joseph francis

The first weekly report is here: t.co/GBeLU1OlAo

isitcredible.com is my new website. You will find a public archive of automated reports on journal articles from across the full domain of human knowledge. You can also use it as a private service to get feedback on your own work.

If anyone out there knows anything about gene-editing, please DM me. I have some questions about this article that go beyond a problematic pathologist's report. 3/3

These are the questions I have asked @nature.com in response.

And this is the article: www.nature.com/articles/s41... 2/3

@nature.com just allowed its authors to retrospectively alter a signed and dated pathologist's report after I pointed out that it contained major errors. 1/3

Given that it is MIT, I am assuming that the brain scan images look like this.

Academia will form these little pockets -- people whose theorizing is outrageous & supported by methods outdated since the 90s -- but once it reaches a critical size those people just review each others papers & grants, form societies, hand out awards etc, like a self-contained parallel society.

I believe that there is a lot of peer-reviewed junk floating around out there, some of it highly cited.

But it is time to cast down the idols. Let’s go! 20/20

Donate to Joseph Francis Help support Joseph Francis by donating or sharing with your friends.

Hence, if you like what I do and can afford it, please make a donation at: paypal.com/donate/?busi....

And if you want to pay me to investigate a study or methodology that you suspect is problematic, my DMs are open. Even if you can’t pay me, I would still be interested! 19/20

For my sins, then, I am here on my hill in Wales, an “independent scholar.” The University of Birmingham has been kind enough to give me an institutional affiliation, but it does not provide any funding. 18/20

Why so quick? Because I have a bad attitude: I’m critical, aggressive, don’t respect my superiors, etc. This meant that I realized early on that I was ill-suited to academia, where disobedience is not often rewarded. 17/20

In the case of ADH (2015), people have been reading and citing it for a decade, accepting it because it had been published in a top journal, etc.

Nonetheless, I first looked at on Saturday afternoon, and it took me about 24 hours to see the problem. 16/20

And, finally, I explain how junk articles persist: the incentives that researchers face to use trendy new methods; the dysfunctional peer-review system that rubber stamps them as truth; the journals that are unwilling to retract articles even when they are shown to be junk. 15/20

Indeed, I argue, the “Synthetic Control Method” is so flimsy that economists do not even need to be aware that they have p-hacked their research design. It can happen by default because everything is left to subjective choice. 14/20

In a second new paper, I then use the case of a recent paper in the Journal of Political Economy to show how the method that ADH use has such researcher degrees of freedom that it is wide open to abuse. github.com/joefrancis50... 13/20

ADH then justify this choice by saying that it did not affect their identification strategy, but––worryingly––do not seem to be aware that it has implications for their causal inference strategy. 12/20

The risk is that ADH have excluded countries that, like West Germany, did poorly in the 1990s, thereby ensuring that West Germany’s poor performance would seem significant compared to the comparators they had selected. 11/20

Furthermore, I do not think the problems in ADH (2015) stop with data handling. This footnote is a major red flag, in my opinion. It suggests that they may have cherry-picked their donor pool to suit their hypothesis. 10/20

The p-value calculated from the permutation test goes up from 0.059 to 0.118.

I do not believe ADH’s results would have been published if they had used the correct variable of interest. As such, their article probably ought to be retracted (cc. @retractionwatch.com‬). 9/20

I have also replicated ADH (2015) using real GDP per capita data from the World Bank and the German national accounts. Otherwise, the data are the same as in the original. And the results aren’t great. 8/20

In the paper, I demonstrate this by comparing their data to the current OECD data, but really you can see it just by looking at their chart above.

It’s obvious that West Germany’s real GDP per capita could not have grown at 8% per year from 1860 to 1990. 7/20

ADH (2015) shows how dysfunctional the current system is.

As I detail in the first of two new papers, they made a simple data-handling error. Rather than use real GDP per capita, they accidentally used nominal GDP per capita: github.com/joefrancis50.... 6/20

Yet peer review is broken. It only encourages conformity and reinforces hierarchy, as well as wasting everyone’s time. www.experimental-history.com/p/the-rise-a... 5/20

Why did no one notice? I fear that it is because scholars are not encouraged to look for such mistakes. Peer review is supposed to be the arbiter of truth. If something is published in a top journal, it must be right. The dominant epistemology assumes that peer review works. 4/20

In other words, people have been reading and citing ADH (2015) for a decade without, it seems, anyone stopping to think “Oh wait, West Germany’s real GDP per capita probably didn’t grow by 8% per year from 1960 to 1990. That would be a miracle.” 3/20

To put this in perspective:

- ADH (2015) was written by Ivy League professors.
- It was published in the prestigious American Journal of Political Science.
- It has 1,504 citations on Web of Science and 3,415 on Google Scholar.

onlinelibrary.wiley.com/doi/abs/10.1... 2/20

Abadie, Diamond, and Hainmueller (2015) is where the replication crisis comes to economics and political science.

Simply put, this chart is impossible. It implies that West Germany’s real GDP per capita grew by almost 8% per year from 1960 to 1990. But it clearly didn’t… 1/20

p.s. In response to popular demand, here is the distribution of the white population in North and South by wealth brackets.

Been on it for a bit, tbh.

The paper is here: raw.githubusercontent.com/joefrancis50....

And it is the fifth in a series of papers, which you can find linked here: bsky.app/profile/joef....

All feedback is very welcome. 12/12