Björn Siepe

*Thanks to my collaborators @fbartos.bsky.social, @boulesteixlaure.bsky.social, @danielheck.bsky.social, @aaronpeikert.bsky.social, Alexandra Sarafoglou & Samuel Pawel.

Diagram showing four phases of methodological research (Theory, Exploration, Systematic Comparison, Evidence Synthesis) with an arrow indicating that preregistration usefulness increases from early to late phases. Each phase lists its aim, elements, outcome, and an example from factor retention research.

Does it make sense to preregister simulation studies?
This question has sparked a lot of debate.

▶️We* work through the why, when, and how
▶️We discuss different phases of methodological research to clarify where preregistration might (or might not) add value

📝 Preprint: doi.org/10.31234/osf...

Congratulations! :)

Call for Papers: Psychometrika Special Issue - Psychometric Society Data Intensive Methods in Psychometrics

Excited to share a new call for papers for a special issue in Psychometrika focused on Data Intensive Methods in Psychometrics that I'll be guest editing with @kyliegorney.bsky.social, @jmbh.bsky.social, @leonievogelsmeier.bsky.social, and Ben Domingue: www.psychometricsociety.org/post/call-sp...

WARN-D machine learning competition is live » Eiko Fried If you share one single thing of our team in 2026—on social media or per email with your colleagues—please let it be this machine learning competition. It was half a decade of work to get here, especi...

After 5 years of data collection, our WARN-D machine learning competition to forecast depression onset is now LIVE! We hope many of you will participate—we have incredibly rich data.

If you share a single thing of my lab this year, please make it this competition.

eiko-fried.com/warn-d-machi...

sure!

To combine some of the answers: I use Todoist, Notion, and Obsidian, all free. Todoist for fast capture of ToDos (works extremely well on mobile), Notion for project management, and Obsidian for longer form content/notes/drafts. Might be overkill to use 3 tools, but this separation works well for me

This sounds like a very interesting and ambitious project! Good luck with finishing it

Thanks for the shout-out! Here's a summary of the database. I'm sure that many of the datasets could be useful for teaching multilevel modeling (but we have no therapy RCT data, unfortunately)

bsky.app/profile/bsie...

🚨Model Checking for Vector Autoregressive Models 🚨

In a new preprint, @joranjongerling.bsky.social, @bsiepe.bsky.social, @sachaepskamp.bsky.social, Lourens Waldorp and I provide a tutorial on model checking for Vector Autoregressive (VAR) models: osf.io/preprints/ps...

Meeting the Bare Minimum: Quality Assessment of Idiographic Temporal Networks Using Power Analysis and Predictive-Accuracy Analysis - Yong Zhang, Jordan Revol, Ginette Lafit, Anja F. Ernst, Josip Razu... The network theory of psychopathology inspired clinicians and researchers to use idiographic networks to study how symptoms of an individual interact over time,...

From my master’s thesis to my first PhD project — excited to share that this work (together with @jordanrvl.bsky.social, @ginettelafit.bsky.social, Anja Franziska Ernst, Josip Razum, Eva Ceulemans, and @bringmannlaura.bsky.social) is now published in AMPPS!

Link: doi.org/10.1177/2515...

While it's certainly becoming less common, it's still a thing in Germany and Austria. From anecdotal evidence, this is particularly the case in medicine

Makes sense, thx! I'll make a note to link docs & metadata more clearly
For now, the best way to obtain the time series relevant to you is probably to download all datasets (still manageable) and filter them yourself. In the long term, with more data, we will work to enable more advanced filtering

Amazing, that's great to hear! Feel free to let me know if you or hack participants have any feedback

Thank you!
Yes, that refers to the maximum (see here: openesmdata.org/docs/data/#n...). The number of observations in my first post (> 740k) refers to actual non-missing observations.
We only used "time points" for brevity/consistency, but I agree this could be confusing & I'll likely change it

I see, that makes sense! I'll note it down on our list for future improvements

Simulation studies have a conflict of interest problem. The same team:
- develops a new method
- designs a simulation study to evaluate it
However, the new method has to show good performance to get published.

We propose living synthetic benchmarks to address the issue (doi.org/10.48550/arX...).

Thanks for exploring openESM!
Do you mean the dates and locations at which data were collected for each dataset? If so, this information has not yet been included because it was often not clearly available. However, we do intend to add more metadata on the details of data collection in the future

Another idea could be to write a consortium paper. For instance, everyone who contributes data could be included in a comprehensive paper on the database. I'm very curious to hear other ideas besides those relating to funding and awards

I also hope that DBs can achieve that! I'm also still unsure how to best incentivize sharing & documentation, both as a scientific community in general, and as a DB maintainer in particular. I suppose that the broad adoption of DBs would considerably increase citations of datasets, which could help

Thank you for sharing, Shirley! :)

While study-level CIs differ, this made no difference for our overall results & pooled effect, so we kept this visualization

Thank you!
We used a 2-step aggregation approach to get study-level effects & CIs here (as recommended in the package we used), but also provide an alternative visualization with shrinkage-based estimates & CIs in the online supplement

We are deeply grateful to everyone who shares their ESM data.
Thanks to @jmbh.bsky.social, @matzekloft.bsky.social, @anabelbuechner.bsky.social, @yongzhangzzz.bsky.social, @eikofried.bsky.social, @danielheck.bsky.social for collaborating on this huge effort - we look forward to your feedback!

A figure showing the software components of the openESM database in two large boxes: Front-end (on the left), and back-end (on the right). The front-end contains documentation and the website with search and is written with html and javascript. The backend contains GitHub repositories with code and metadata that serve the website. They are related to Zenodo repositories with data and metadata, and our helper software packages in R and Python. We use TSV files for ESM data and JSON files for metadata storage.

Further details:
▶️Search and filter datasets on openesmdata.org
▶️Auto-generate R/Python code
▶️(Meta-)data are stored on Zenodo with DOIs
▶️Metadata and software on GitHub enable community contributions
▶️Contribution guidelines allow further database extensions so that openESM can continue to grow

A forest plot containing the point estimates, uncertainty, and additional information about the within-person correlations of positive and negative affect across datasets. The figure shows the dataset names on the left side, estimates in form of a point and uncertainty lines in the middle, and the number of individuals, average timepoints, and correlation for each dataset on the right-hand side.

▶️Our example analysis shows how to use openESM: We estimated within-person correlations between positive and negative affect across 39 datasets (>500K observations)
▶️We find a robust negative correlation (−0.49 [-0.54, -0.42]) and outline ideas for future research building on this

The image shows two code blocks, horizontally next to each other. The left one shows R code, the right one shows Python code. Both contain code to load the openesm package in the respective language, list_datasets() to get an overview of all datasets, download an individual dataset using get_dataset(), and obtaining citation information for this dataset. Further explanations are available in the paper and our online documentation.

▶️In our introduction paper, we outline why large-scale analyses are important for substantive, design, and statistical research
▶️To make such research easier, we provide rich metadata for each dataset, plus dedicated R and Python packages to easily access and handle the data

▶️Individual ESM studies are often limited in size and diversity of population and measures
▶️Open data are scattered across repositories in different formats, impeding research into robustness, generalizability, and heterogeneity
▶️We aim to change this to enable large-scale, cumulative ESM research

We built the openESM database:
▶️60 openly available experience sampling datasets (16K+ participants, 740K+ obs.) in one place
▶️Harmonized (meta-)data, fully open-source software
▶️Filter & search all data, simply download via R/Python

Find out more:
🌐 openesmdata.org
📝 doi.org/10.31234/osf...

Congrats, Omid! 🎉