Raphael Nishimura

One of my many pet peeves is people who denote survey sample sizes by N.
In surveys statistics, because we are dealing with finite population inferences, we reserve the notation N for population size and use n for sample size.

Outdoor photo of the Convergence sculpture at ISR during a snowfall; a white text box reads “INSTITUTE FOR SOCIAL RESEARCH, Snow Flurries on Convergence.”

Interior view of ISR's atrium with hanging plants and glowing pendant lights; white text box reads “INSTITUTE FOR SOCIAL RESEARCH, Atrium Glow.”

Exterior photo of Perry Building with clear blue sky and red brick facade; white text box reads “INSTITUTE FOR SOCIAL RESEARCH, Perry Brick Red.”

Scenic rooftop view from ISR’s 6th floor showing a blue flag with a yellow M flying above Ann Arbor rooftops; white text box reads “INSTITUTE FOR SOCIAL RESEARCH, View From the 6th Floor.”

The Shades of ISR Pantone ™️ palette 💙 💛

That's one Meng's point in his 2018 paper, isn't it?

Oh yeah, for sure, that's quite clear there. My point is that, as a general rule, statistically speaking, it just doesn't make sense to me, despite the empirical evidence. Bias in surveys doesn't typically scale with sample size, which is what the rule suggests.

I always thought this rule that the total error is 2x reported MoE nonsensical, because bias in surveys doesn’t typically scale with sample size.
In fact, we usually see the opposite: smaller, more carefully designed prob samples tend to present smaller total error than large nonprob samples.

YouTube video by Survey Research Center, University of Michigan We Are SRC

The Survey Research Center at the Institute for Social Research of the University of Michigan turns 80 years old this year. Very proud to be part of this history!
youtu.be/giSsP7o4PZg?...

E o mais importante, a descrição metodológica da pesquisa:

Cenários de 2° turno: +

Cenários de 1° turno com Michelle: +

Resultados da pesquisa Meio Ideia, primeira pesquisa pré-eleitoral nacional de 2026 registrada no TSE (BR-06731/2026).
Pesquisa telefônica com 2 mil entrevistas realizadas entre 08/01 e 12/01.
Perguntas espotânea e estimuladas com cenários de 1° turno com Tarcísio e Flávio): +

Sorry, I have just seen this.
I have just answered there.

I'm just a mere survey statistician, I leave this sort of political analysis to my polisci colleagues 😛

I think most of the major pollsters that use household, face-to-face data collection is using some sound and robust methodology.

It's because, as in many other media outlets, they don't have anyone there with a good grasp of survey methods and they almost buy this pollster shameless self-promotion, mostly based on cherry picking and fallacious metrics.

As usual, regardless of the poll result, I'd always take any of their polls with a huge grain of salt. Maybe all the salt in the oceans? 😅

It essentially depends on two things: the amount of measurement error and how correlated the auxiliary variable is with the survey variables.
Just as with missing data (e.g., coverage, sampling or nonresponse), the math doesn't care about what is the source of error!

Here are the slides: aapor.confex.com/aapor/2025/m...

I did a whole simulation on this and presented at AAPOR earlier this year!

Merry Christmas! 🎅🎄

At this point, people shouldn't be surprised by this. Their methodology is crap, so you get this kind of unstable and unreliable estimates. You all should just stop fooling yourselves.

The point here is: they were relying A LOT on weighting to adjust their heavily skewed sample towards Trump voters. Is their methodology still producing this kind of sample? And can the weights sufficiently address this issue?

So, I have no idea whatever kind of adjustment they are making to obtain the result that got them the title of "2020 America's most accurate pollster". And why can't we replicate these weights with their microdata following what is described in their stated methodology? +

I tried several different versions of weighting by those variables, but in none of them I was able to replicate their final estimate of Biden 52 Trump 47. In fact, in none of the weights, I was even able to get Biden ahead of Trump. The closest I got was Biden 49 Trump 50. +

Also, I tried to replicate their weight according to the stated methodology. There is a discrepancy here because in one it's mentioned they weighted by 2016 recall vote, but not in the other. From other polls of the same pollster, I'm pretty sure they did weight by recall vote. +

On one hand, that's good: they are doing what they are supposed to be doing. On the other hand, if your sample data is usually THAT skewed that you are relying that much in the weights, that can be very problematic if for any given race the weights don't manage to correct that.

The unweighted result is Trump 66% Biden 33% (!!!). Recall 2016 vote is 70% Trump 30% Clinton. The weights are doing A LOT of work here! +

Below is their 2020 national poll methods per Roper Center. Not much different to what a lot of other pollster are doing. However, if you look at the polling microdata also in Roper...+

Let me bring that information here 🧵

Congrats!!

@badsurveyq.bsky.social