Nate Phillips

W/ @dillonmwong.bsky.social, @courtlandhyatt.bsky.social, @drlynam.bsky.social, and @jdmiller.bsky.social

Continuing to think through the degree to which “individual difference” domains explain variance in EXT above and beyond personality traits through additive and interactive effects feels like an important first step to building stronger, empirically driven theories of EXT

Additive and Interactive Relations of Personality and Cognition With Externalizing Behaviors - Nathaniel L. Phillips, Nathan T. Carter, Kevin M. King, Courtland S. Hyatt, Max M. Owens, Donald R. Lynam... Personality and cognition offer robust frameworks to understand the individual differences associated with externalizing behaviors. However, these literatures h...

In many ways, this serves as an extension of our paper at CPS where we asked the same basic question but with personality and cognitive aptitude

journals.sagepub.com/doi/abs/10.1...

Happy to see this now out at JRP

For this one, we took a swing at integrating two historically siloed literatures that focus on the individual differences related to externalizing behaviors: personality and formidability

Paper on statistical power necessary for interaction effects
doi.org/10.1177/2515...

Tl/dr: Most estimated effects did not meet our preregistered sig. threshold, except Lack of Premeditation showed a robust, positive association with time spent gambling. Within-person momentary affect variables’ relations with gambling behaviors were weak and varied considerably among participants

Happy to share that this has just been accepted over at Journal of Psychopathology and Behavioral Assessment

w/ @vizecolin.bsky.social, Kate Collison, Michael Crowe, @drlynam.bsky.social, & @jdmiller.bsky.social

Richard McElreath: It must not be overlooked that junior researchers DO NOT TRUST US. We, the directors, are a big part of the problem. We made this system, we remake it every year, and we benefit from it. What can we do to credibly signal our commitment to reform a corrupt research culture? My conversations with junior scientists in the society has taught me that directors are too often either indifferent or hostile to science reform. We cannot hope to convince our prize winning colleagues. Their egos are immune. But we can replace retirements with researchers who care more about integrity than their own prestige. This is important both for earning the trust of the junior researchers who really do the research in the MPG and for attracting excellent future directors and starting to earn the trust of the public. So I suggest two strong signals to our junior researchers (and the public): (1) we will reform recruitment and promotion at all levels to eliminate proxies like citation counts and journal brands in favor of reliability and sustainability; (2) we will make open science skills a core part of scientific training, through the graduate schools at a minimum, as conditions for the central funding. The most ambitious thing we could do, as hinted at in item 5 above, is to meaningfully invest in metascientific research. As the largest basic research organization in the world, the MPG is uniquely suited to studying research and its products from a broad perspective that includes the humanities, the sciences, and policy. Governments are already involved in science reform. Someone should study it in an organized and sustained way.

The Max Planck Society has begun an exploratory round table for open science. We are drafting some recommendations to leadership. Still a long way to go! But here are my notes on the most recent draft, just so you all know how I am trying to steer things.

This paper is now out in print:

journals.sagepub.com/doi/10.1177/...

TLDR still holds.

Perils of Partialing: Can Scholars Predict Residualized Variables' Nomological Nets? Objective Partialing is a statistical procedure in which the variance shared among two or more constructs is removed, allowing researchers to examine the unique properties of the residualized, parti...

Excited to share a Registered Report in J. of Personality looking at the “perils of partialing” – led by the Bluesky-less Leigha Rose with @drlynam.bsky.social and me. (1)

onlinelibrary.wiley.com/doi/10.1111/...

We just preprinted a huge meta-meta-analysis examining the effects of exercise on cognition, memory, and executive function

In short
- 2239 effect sizes
- extreme between-study heterogeneity
- extensive publication bias
- some subgroup/exercise-specific effects

More below (doi.org/10.31234/osf...)

Very useful table of insufficient examples vs. best-practices for statistical reporting!

Will definitely point some of my colleagues to it! 📈

Massive and important undertaking by @andrew-cast.bsky.social, @drlynam.bsky.social, and co in estimating when interaction effects stabilize in linear regression

Tldr: Under realistic conditions for psych research, N = 3,800

When do interaction/moderation effects stabilize in linear regression?: https://osf.io/35t84

Exciting news from the Academy of Psychological Clinical Science--a joint effort to enhance open science training in clinical psychology!

If you are interested, you can the papers mentioned in the email here:
Van Til et al. osf.io/h34jg/files/...
OSC paper (Lynam et al.) osf.io/preprints/ps...

Hey Philipp, what’s your email? I can send it your way

Love to see a registered report that reinforces why the registered report is such a valuable tool

Interactions are difficult to detect in field studies as they are typically tiny--very small to start with and made smaller by the joint unreliabilities of the components. Here, we find some but the contribution to explained variance is negligible. Call off the search. It is not worth the effort.

Pretty excited about this one. In this paper, we discuss the replication/credibility crisis, the factors that contribute to it, and clinical psychology's slow (really slow) progress in dealing with it. We offer a competency-based fraemwork for improving our training of future scholars.
1/2

Rigorous science is transparent science.

New paper led by @drlynam.bsky.social on the need for more training in and engagement with open science practices in clinical psych programs. It has been difficult to make progress due to a variety of barriers, including students working in labs uninterested or hostile to these approaches.

Just accepted from @vizecolin.bsky.social and myself. We coded Open Science practices (preregistration, RRs, open data, and open code) from 2021 to 2024 in two personality disorder journals (JPD, PDTRT) and three personality journals *JOP JRP, and EJP).
osf.io/preprints/ps...
1/5

Lol 100 percent. I feel similarly when diving into this literature and posted this a little while ago hoping for some clarity (and got crickets)

bsky.app/profile/nphi...

A review of spline function procedures in R - BMC Medical Research Methodology Background With progress on both the theoretical and the computational fronts the use of spline modelling has become an established tool in statistical regression analysis. An important issue in splin...

Not sure if this is the type of work you're looking for, but I read this one recently and found it helpful! Interested to see what other folks recommend bmcmedresmethodol.biomedcentral.com/articles/10....

Additive and Interactive Relations of Personality and Cognition With Externalizing Behaviors - Nathaniel L. Phillips, Nathan T. Carter, Kevin M. King, Courtland S. Hyatt, Max M. Owens, Donald R. Lynam... Personality and cognition offer robust frameworks to understand the individual differences associated with externalizing behaviors. However, these literatures h...

𝐀𝐝𝐝𝐢𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐈𝐧𝐭𝐞𝐫𝐚𝐜𝐭𝐢𝐯𝐞 𝐑𝐞𝐥𝐚𝐭𝐢𝐨𝐧𝐬𝐨𝐟 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥𝐢𝐭𝐲 𝐚𝐧𝐝 𝐂𝐨𝐠𝐧𝐢𝐭𝐢𝐨𝐧 𝐖𝐢𝐭𝐡𝐄𝐱𝐭𝐞𝐫𝐧𝐚𝐥𝐢𝐳𝐢𝐧𝐠 𝐁𝐞𝐡𝐚𝐯𝐢𝐨𝐫𝐬 | "Although interaction effects were detected, they were small and practically negligible in their explanation of variance in externalizing behaviors" journals.sagepub.com/doi/10.1177/...

This imagery is the kind of thing you see in video games to let you know you’re in a totalitarian city or country.

Models as Prediction Machines: How to Convert Confusing Coefficients into Clear Quantities Abstract Psychological researchers usually make sense of regression models by interpreting coefficient estimates directly. This works well enough for simple linear models, but is more challenging for more complex models with, for example, categorical variables, interactions, non-linearities, and hierarchical structures. Here, we introduce an alternative approach to making sense of statistical models. The central idea is to abstract away from the mechanics of estimation, and to treat models as “counterfactual prediction machines,” which are subsequently queried to estimate quantities and conduct tests that matter substantively. This workflow is model-agnostic; it can be applied in a consistent fashion to draw causal or descriptive inference from a wide range of models. We illustrate how to implement this workflow with the marginaleffects package, which supports over 100 different classes of models in R and Python, and present two worked examples. These examples show how the workflow can be applied across designs (e.g., observational study, randomized experiment) to answer different research questions (e.g., associations, causal effects, effect heterogeneity) while facing various challenges (e.g., controlling for confounders in a flexible manner, modelling ordinal outcomes, and interpreting non-linear models).

Figure illustrating model predictions. On the X-axis the predictor, annual gross income in Euro. On the Y-axis the outcome, predicted life satisfaction. A solid line marks the curve of predictions on which individual data points are marked as model-implied outcomes at incomes of interest. Comparing two such predictions gives us a comparison. We can also fit a tangent to the line of predictions, which illustrates the slope at any given point of the curve.

A figure illustrating various ways to include age as a predictor in a model. On the x-axis age (predictor), on the y-axis the outcome (model-implied importance of friends, including confidence intervals). Illustrated are 1. age as a categorical predictor, resultings in the predictions bouncing around a lot with wide confidence intervals 2. age as a linear predictor, which forces a straight line through the data points that has a very tight confidence band and 3. age splines, which lies somewhere in between as it smoothly follows the data but has more uncertainty than the straight line.

Ever stared at a table of regression coefficients & wondered what you're doing with your life?

Very excited to share this gentle introduction to another way of making sense of statistical models (w @vincentab.bsky.social)
Preprint: doi.org/10.31234/osf...
Website: j-rohrer.github.io/marginal-psy...

This one and McCabe et al 2022 from @kevinmking.bsky.social's group are both awesome in thinking about the interpretability of these types of effects

For those interested, here is a link to a new power paper:

Hancock, G. R., & Feng, Y. (2026). nmax and the quest to
restore caution, integrity, and practicality to the sample size planning process. Psychological Methods.

yifengquant.github.io/Publications...

Really appreciate the authors’ efforts to differentiate these practices. I’ve definitely seen the term “preregistration” used to describe each of these three (registration, protocol, analysis plan) in isolation of one another, so it’s great to have a framework to address these jingle-jangle issues