Rich Lucas

And I, too, would think I was beating a dead horse if it weren't for the fact that top journals keep publishing simple lag-1 CLPMs with no controls (meta-analytic CLPMs are still CLPMs): www.apa.org/pubs/journal...

Reposting this thread about another recent preprint that discusses some of the reasons why: bsky.app/profile/rich...

Rates of significant effects are also arguably too high with the RI-CLPM. Individual effects are about half as likely to be significant with the RI-CLPM as with the CLPM, but you still get at least one significant lagged effect in 61% of models.

We already know that lagged effects in CLPMs are likely to be upwardly biased, but just how easy is it to find significant effects? Way too easy. I tested CLPMS in 100 randomly selected pairs of correlated variables and found significant effects in 98 of them. New preprint: osf.io/preprints/ps...

Anyway, we welcome any thoughts or suggestions you have about the paper! [15/15]

Patterns of stability over increasingly long lags for the Big Five personality traits. The slope of these stability coefficients is very shallow.

Also: state variance is not just measurement error, so using latent variables with the CLPM or RI-CLPM won't always fix this (though it helps). Even with multiple-item Big Five scores modeled using latent traits, the same pattern emerged. [14/x]

Our analyses also suggest that once you account for state variance, stabilities of variables included in these panel studies are extremely high: Median 1-year stability = .90. This suggests that there is often very little change occurring that could be accounted for by a lagged effect [13/x].

Our analyses suggest that the CLPM and RI-CLPM would be the most appropriate model for just 4% of variables we examined; the other 96% were divided equally between ARTS and STARTS [12/x]

Implied and actual stability coefficients for a 10-year ARTS model. The implied and actual stability coefficients are very similar.

What type of model can account for these patterns? Models that include a state component (like the STARTS or even the ARTS, which drops the stable trait). You can reproduce actual patterns of stability even without assuming the existence of a stable trait (this plot is for the ARTS) [11/x]

Actual stability coefficients across increasingly long lags for over 400 variables. Most stabilities decline slowly without ever reaching an asymptote.

It turns out it's very common. Here are the plots of stability over increasingly long lags for about 400 variables from a large panel study. For anything less than almost perfect short-term stability, stability coefficients should reach an asymptote long before 22 years; very few do [10/x]

One pattern it can't handle is when short-term stability is moderate, medium-term stability is just slightly lower than long-term stability, and there is no clear asymptote with increasingly long lags (as is true with life satisfaction and health). How commons is this pattern? [9/x]

Why doesn't the RI-CLPM just estimate a lower asymptote (corresponding to less stable trait variance)? It turns out that the RI-CLPM is quite limited in the types of patterns of stability with which it is compatible (won't go into details here, but paper does). [8/x]

Implied and actual stability coefficients for a 10-wave RI-CLPM. Evidence of misspecification emerges at longer lags.

This gets even worse with more waves of data. In our experience, this is a pretty common pattern: If you have more than 4 or 5 waves, the RI-CLPM overestimates long-term stability (and often underestimates medium-term stability). This misfit can also lead to bias in estimates [7/x]

Implied and actual stability coefficients for the RI-CLPM. These values are close to one another with a hint of misspecification at long lags.

The RI-CLPM does better, but even here, there is a hint of misspecification (see how stability coefficients continue to decline even after the RI-CLPM predicts an asymptote) [6/x]

Implied and actual stability coefficients over five years for life satisfaction and health. The implied stability coefficients are much lower than the actual stability coefficients.

Here's a plot of actual stability coefficients and those implied by a fitted CLPM with life satisfaction and health. It's really bad...CLPM dramatically underestimates stability. This misspecification introduces bias, which can be severe [5/x]

There are already lots of good papers comparing these models from a causal inference perspective, but we focus more on typical patterns of stability in real data and how well they match with the patterns of stability implied by these models. Why is this useful? Consider the CLPM. [4/x]

TL/DR: More reasons why the standard lag-1 CLPM is really bad, but also some important concerns about models that assume the existence of a stable trait (like the RI-CLPM). Our analyses suggest that we should be paying more attention to state variance when modeling these effects [3/x]

We try to do three new things. 1. Focus explicitly on model misfit and its implications for bias. 2. Conduct simulations showing how specific forms of misfit affect bias. 3. Analyze the longitudinal structure of over 400 variables to see what type of misfit is likely for specific models. [2/x]

Interested in models used to estimate lagged effects in panel data? We (@rebiweidmann.bsky.social, Hyewon Yang) have a new paper looking at patterns of stability and their implications for bias and model choice: osf.io/preprints/ps... [1/x]

If you're looking for a good quarto extension to do APA-style documents, this one does it! Seems to have everything papaja had for Rmarkdown and more. Really useful and easy to implement (including great export to docx): wjschne.github.io/apaquarto/

Will get right on that (checks to see whether @syeducation.bsky.social has something I can use as a template that I can pass on to @jnfrltackett.bsky.social )

Abstract: The cross-lagged panel model (CLPM) is an analytic technique used to examine the reciprocal causal effects of two or more variables assessed on two or more occasions. Although widely used, the CLPM has been criticized for relying on implausible assumptions, the violation of which can often lead to biased estimates of causal effects. Recently, a triangulation method has been proposed to identify spurious effects in simple CLPM analyses (e.g., Sorjonen & Melin, 2024b). We use simulations and a discussion of the formulas underlying regression coefficients to show that this method does not provide a valid indicator of spuriousness. This method identifies true causal effects as spurious in realistic situations and should not be used to diagnose whether a causal effect estimated from the CLPM is spurious or not. There are clear reasons to doubt causal estimates from the CLPM, but the results of the triangulation method do not add information about whether such estimates are spurious.

The problems w/ two-wave cross-lagged models are becoming widely known. Some proposed a "triangulation method" (2nd screenshot) to identify spurious effects in such designs

We show that this test doesn't work in plausible situations. It shouldn't be used.

journals.sagepub.com/doi/10.1177/...

Me, trying to rerun analyses from a years-old paper for a request from a meta-analyist

Collabra: Psychology is looking for new associate editors for the Social Psychology section. If you're interested, please apply! rug.eu.qualtrics.com/jfe/form/SV_...

My first Skeet! Happy to announce that our paper on life events and life satisfaction is now published at the European Journal of Personality (w/ @dingdingpeng.the100.ci, @richlucas.bsky.social, and @drichter77.bsky.social)! journals.sagepub.com/doi/10.1177/... (preprint: osf.io/preprints/ps... ).

Oh no; that is so sad. Gert was so warm and welcoming when I started working with the SOEP data, and supportive and encouraging ever since. So much of the way that personality psychologists work now (with lots of analysis of panel data) is due to Gert's early support of psych measures in the SOEP.

Research suggested a response styles bias in people's repeated self-reports (e.g. daily diary studies, EMA). We tested this and found evidence against a broad bias. Open access: journals.sagepub.com/doi/10.1177/...
with @richlucas.bsky.social, Elise Kalokerinos, Noelle Rensing, and Iris Schneider.

The second thing is that for the second time in a row (see below), our post-publication critique was received really well by the original authors. Could this be a new norm? We have one more coming soon, so hopefully it is :) bsky.app/profile/rich...

They were also able to present some of this work at their undergrad research fair, and after preparing that presentation, our undergrads now have a really good understanding of some of the problems with difference scores

This was a great opportunity for our undergrad assistants to see that just because something is published, that doesn't mean there aren't questions left and that we can address those questions with new analyses and data