A major survey ran for over a year before anyone realised the data was wrong. When the issue surfaced, the polling firm responsible did the right thing and retracted it. The question worth asking is how fraudulent respondents got into the data and why nobody caught it sooner.
In early 2026, the data behind the Bible Society’s Quiet Revival report was retracted after fraudulent respondents were identified in the sample. The 2024 survey had claimed church attendance in England and Wales was rising sharply, particularly among the under-25s, with monthly attendance in that age group supposedly climbing from 4% in 2018 to 16% by 2024.
The decision to withdraw the findings and acknowledge the issue reflects well on how seriously reputable firms take data integrity. But the episode raises important questions for anyone commissioning research. The defences existed, they simply were not running. And it took more than a year for the problem to surface.
The poll’s findings were promoted for around 15 months. During most of that period, social scientists working on UK religion data had been raising concerns that the numbers looked unusually high. The Church of England’s own attendance figures over the same period were still tracking below pre-pandemic levels, according to CNN. However, the Quiet Revival story continued to circulate in media coverage and public commentary regardless.
It turns out that correcting a flawed survey takes much more effort than publicising it in the first place. When a newspaper retracts a front-page story weeks or months later, the retraction is a small paragraph on page 7 that hardly anyone sees. Anyone commissioning polling should understand this. Your retraction will reach a fraction of the audience that saw the original story, which is why getting the methodology right from the start matters so much.
Nobody is claiming AI was involved in this case. The cause was fraudulent respondents and a process gap, not bots. But the incident has occurred at a time when the deeper vulnerability of online opt-in surveys is becoming harder to ignore across the entire industry.
Courtney Kennedy at Pew Research has written about a pattern she calls positivity bias in opt-in surveys. Fraudulent respondents tend to answer yes to qualifying questions because saying yes is what gets them through the screener and into the survey pool. The more screeners passed, the more surveys completed, the higher the payout.
There’s also a demographic angle. Under-30s are difficult to recruit for legitimate polling, which means quotas in that bracket stay open longer and pay more. Fraudulent actors respond to that by presenting themselves as young. The supposed growth in the church data was concentrated in the under-25s, which is also the group most exposed to fake answers. This isn’t a problem specific to any one study. It applies to any survey relying on the same online panel infrastructure.
Sean Westwood at Dartmouth has put the broader point plainly. The assumption that a coherent set of survey answers must have come from a real person no longer holds.
His 2025 PNAS paper provides the evidence. AI bots passed 99.8% of standard online survey quality checks across 6,000 trials. Between 10 and 52 fake responses was enough to flip the apparent leader in major national polls.
While AI was almost certainly not the cause in the church poll case, the tools for generating plausible fake survey responses are getting cheaper and more accessible at a pace detection tools aren’t matching. This is an industry-wide challenge, not a problem unique to any single firm.
Online surveys aren’t useless. For pulse work, consumer tracking, and attitude monitoring, they remain a valuable tool.
The calculation changes when results will be published, quoted by journalists, cited in policy work, or used for decision-making where accuracy matters. In those situations, the methodology question has become more significant than it was a few years ago.
Were respondents verified by a person, or only screened by software? Was there a human in the loop to verify respondent authenticity and engagement? Or was the process fully digital?
CATI has its own limitations, but a trained interviewer on the line is a verification step that fraudulent respondents and automated tools will struggle to get past. That ability to check who’s actually answering the survey is becoming increasingly valuable.
The Quiet Revival episode doesn’t prove online research is flawed. It demonstrates what happens when the verification step is absent across the broader panel ecosystem, and why even well-resourced firms aren’t immune to the problem. The right response isn’t to abandon online research, but to understand where it carries risk and to build accordingly.
For insights on polling methodology and respondent verification, contact our research team.
Positivity bias is a pattern where fraudulent respondents in online panels answer yes to qualifying questions because saying yes is what gets them through the screener and into the survey pool. Pew Research’s Courtney Kennedy has documented this as a structural problem for opt-in panels. The more screeners a respondent passes, the more surveys they complete, and the higher their payout. The effect tends to concentrate in demographics that are hard to recruit legitimately, such as under-30s.
Yes, and detection is getting harder. Sean Westwood’s 2025 PNAS study found that AI bots passed 99.8% of standard online survey quality checks across 6,000 trials. The same study showed that between 10 and 52 fake responses was sometimes enough to flip the apparent leader in major national polls. The tools for generating plausible fake responses are improving faster than the tools for detecting them, which makes verification at the fieldwork stage increasingly important.
Polls are retracted when a methodological problem comes to light after publication, most commonly fraudulent respondents in the sample, errors in weighting, or sample composition issues that weren’t caught during fieldwork. The Bible Society’s Quiet Revival report was retracted in early 2026 after fraudulent respondents were identified in the underlying 2024 survey. Retractions are the right response when a problem is found, but they typically reach a small fraction of the audience that saw the original findings.
The most important questions are about fieldwork rather than headline numbers. Was the sample drawn from a verified source or an opt-in online panel? Were respondents authenticated by a person, or only screened by software? Were attention checks, consistency checks, and quality control systems actually running during fieldwork? Has the methodology been documented in enough detail for an independent party to review it? Reliable polling stands up to those questions before the results are published, not after.
CATI (Computer Assisted Telephone Interviewing) uses trained human interviewers to conduct surveys by phone. Online panel surveys use pools of pre-recruited respondents who complete surveys digitally, usually in exchange for incentives. CATI is generally slower and more expensive per complete, but a human interviewer on the line is a verification step that fraudulent respondents and automated tools will struggle to get past. Online panels offer scale and speed but rely on the panel provider’s quality controls to keep fraudulent and bot responses out of the sample.
The stakes change when results will be quoted by journalists, cited in policy work, or referenced in public commentary. In those situations, the methodology question matters more than cost or speed. The questions to weigh are whether respondents are human-verified, whether the sample frame is defensible, and whether the fieldwork process produces an audit trail that can stand up to scrutiny if the findings are challenged. For lower-stakes pulse work and attitude monitoring, online panels remain a valuable tool. For polling that will be published and referenced, the verification step is where the risk lives.
