Infectious disease: Key surveys overestimate COVID-19 vaccination rates in the USA

Estimates of COVID-19 vaccine uptake in the USA based on large surveys that are used to guide policy-making decisions tend to overestimate the number of vaccinated individuals, research published in Nature suggests.

In the USA, the CDC (Centers for Disease Control and Prevention) compiles data on national vaccine uptake, but reporting can sometimes be delayed. Surveys that measure attitudes and behaviour towards COVID-19 vaccines can fill a gap when there is a lag in real-time data, and can inform government responses to the epidemic. However, some surveys diverge substantially in their findings.

The authors find that Delphi–Facebook’s COVID-19 symptom tracker (with about 250,000 responses per week) and the Census Bureau’s Household Pulse survey (with about 75,000 responses per week) overestimated vaccine uptake by 17 and 14 percentage points, respectively, in May 2021 compared to a benchmark from the CDC. These overestimation errors go orders of magnitude beyond the statistical uncertainty provided by the surveys. A survey by Axios–Ipsos also overestimated uptake, but by a smaller amount (4.2 percentage points in May 2021)—despite being the smallest survey (about 1,000 responses per week). These findings indicate that bigger is not always better when it comes to datasets, if we fail to account for data quality.

The authors note that COVID-19 vaccine uptake was not the primary focus of any of the surveys. For example, Delphi-Facebook intends to measure changes in COVID-related behaviors over time. However, the bias in estimates of vaccine uptake in the two large surveys indicates that they are not representative of the US adult population. Lack of statistical representativeness may also be causing bias in other survey outcomes they suggest.

The authors suggest that design choices in survey data collection can lead to inaccuracies; for example, the three surveys use different recruitment methods, which may introduce different biases in their estimates. Efforts to measure data quality and improve the accuracy of assessments of vaccine uptake are needed to better inform public policy decisions, the authors conclude.