Jack Fitzgerald

Had a great time presenting my job market paper at the Lindau Nobel Meeting in Economic Sciences! 🔗 : osf.io/d7sqr_v1/

#LINOecon #EconSky

Registration closes July 4! Come join us in-person or online in Amsterdam!

Dutch academics, please share - replication games are coming to Vrije Universiteit Amsterdam on July 19! If you live in the Netherlands, or want a good reason to visit, come take a day to network, hone your skills, and potentially coauthor a paper. Registration: www.surveymonkey.ca/r/Replicatio...

#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is.

Short 🧵

This will likely be a good initiative, conditional on it not substituting for the journal publishing full comments. Publication is an important incentive for reproducibility/robustness analyses and substantive empirical critiques almost always take more than one page.

theoretical also having a real bad year in 2000

those poor y-axes

#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.

The logo for the Journal of Marginally Significant Results naturally features a series of confidence intervals hanging out anxiously close to zero

Do the tables get imaginary numbers?

Email inviting me, a PhD candidate, to be the editor-in-chief of a brand new journal of my choosing.

What academic journal should I start? Wrong answers only

#GDRI_rep Update 4: We have a new report. We reproduced the paper entitled "Parent–teacher meetings and student outcomes" by Islam @ European Economic Review. See below for links to report and author's response.

Thank you @carlislerainey.bsky.social for boosting my job market paper! I’’m a fan of your prior work on this topic.

If you want to learn more, check out the paper and my BIBAP seminar on it tomorrow, 3/12 7 AM CET/5 PM Sydney time.

www.unsw.edu.au/business/beh...

Is There a Foreign Language Effect on Workplace Bribery Susceptibility? Evidence from a Randomized Controlled Vignette Experiment - Journal of Business Ethics Theory and evidence from the behavioral science literature suggest that the widespread and rising use of lingua francas in the workplace may impact the ethical decision-making of individuals who must ...

This is wild to hear. For what it’s worth, in my publication at Journal of Business Ethics, the equivalence testing results were the thing the editors and reviewers liked *most* about the paper. I think equivalence testing is key for making null results more publishable!

doi.org/10.1007/s105...

#5 Here's another recent critique of close elections RDDs.

It's unclear whether there is actually good evidence against potentially fatal precise manipulation around the cutoff in these designs.

I'll be waking up early (7 AM CET) on Tuesday, March 12 to present my job market paper at 5 PM Sydney time! If you're awake too, stop by to hear me talk about equivalence testing, replication-based methods research, and the robustness of null results in economics!

Relatedly, I also develop new equivalence testing methods for testing running variable (RV) manipulation in RDD. In 36 RDD publications, I find >44% of RV density discontinuities at treatment cutoffs can't be significantly bounded beneath 50% upward jumps. 🧵: bsky.app/profile/jack...

#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/

#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵

Our proposal for post-publication review: “our system would add only 1 percent to the total reviewing effort, while providing important perspectives on papers representing more than one-quarter of the citations received by these influential journals.”
statmodeling.stat.columbia.edu/2025/02/26/pp/

The Need for Equivalence Testing in Economics

As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.

Expat in NL here: you would also call it Den Bosch if you had to pronounce ‘s Hertogenbosch often enough

I had a wonderful time presenting my job market paper at RWI Essen! Thank you @jrgptrs.bsky.social and Julian Rose for hosting and organizing!

Additional context: this was just in my paper. Different teams replicated different papers depending on which Games they attended and which software they used. The main results control for Games-software fixed effects to partial out reproducibility difficulty across different replication packages.

I’m a fan of LLMs and use them to program all the time, but some general lessons here. 1: Don’t trust an LLM’s reproduction of a paper; do it yourself. 2: Humans are necessary to ensure reproducibility. 3: For the love of all that is holy, please double-check LLMs’ output. 7/7

We weren’t alone. Reproducibility rates for AI-led teams were over 57 p.p. lower than those for AI-assisted teams, and over 59 p.p. lower than those for human teams. AI-led teams also detect fewer major errors and propose/implement fewer good robustness checks. 6/7

This replication package had *annotated code*. We gave ChatGPT data + scripts that *label the code for Figure 4*, and then asked it to reproduce Figure 4. It didn’t even get close to reproducing Figure 4, even after a mind-melting amount of effort put into prompt engineering. 5/7

Hours were spent trying to get ChatGPT to write code that would produce any plot at all. Eventually, we abandoned R for Python. We switched ChatGPT terminals, considering our current terminal to be a lost cause. We still couldn’t reproduce the figure by the end of the day. 4/7

Our team could only upload the paper and its replication package into ChatGPT along with an image of the figure we wanted to reproduce, try to get ChatGPT (o1 for our Games) to write code that would reproduce the figure, and then run this code in R. This was ✨misery✨. 3/7