The Hidden Problem With Traditional Community Surveys

phone survey blog

Local government is one of the hardest jobs there is. The decisions civil servants make include whether to fund police overtime or a new park, which streets get repaved this year, and what the tax rate should be. Decisions like these should include community voices. 

In practice, community voices come from a vocal group of residents who attend meetings, follow social media updates, and eagerly share their views. Government staff have a name for them: the same ten people. Though these voices are essential and engaged, they are rarely representative of the community as a whole (Einstein, Palmer & Glick, 2019). It isn’t hard to see why. Attending a midday hearing requires free time, a flexible schedule, and enough familiarity with local government to know the meeting is happening at all. So the room fills with a small, consistent group, a group that skews more affluent, older, and more politically engaged.

Give the same ten people the microphone and their input can point leaders in directions that the full community wouldn’t support. This is borne out in research on public hearings: of the residents who do show up, 64 percent came specifically to oppose what is being discussed, while only a small minority came to support it (Einstein et al., 2022; Einstein, Palmer & Glick, 2019). And the same ten people often have different opinions than the community around them (Martin & Venugopal, 2026).

Local government surveys were made to solve this exact problem. For decades, mail and telephone surveys kept this promise, reaching past the same ten people in the room. However, fewer and fewer residents answer those surveys each year (Meyer, Mok & Sullivan, 2015). The methods stayed the same, and the surveys still produce results: charts, percentages, and margins of error. But the people underneath those numbers have been shrinking for years, and have shrunk unevenly.

Traditional Surveys Were Built for a Different Era

When we say a “traditional survey,” we mean a study that assembles a fixed list of names and addresses, sends a survey to that list by mail or by telephone, and waits to see who replies. This approach was designed for a world where nearly every household had a landline and answering a call from a stranger was normal. Two things have changed:

The first is the number of responses. Phone surveys used to hear back from roughly a third of a community. Today, it is under six percent. Mail survey response rates have fallen in similar fashion (Kennedy & Hartig, 2019). Low response rates don’t necessarily compromise a survey’s capacity to represent a community, but they make a survey more reliant on statistical adjustments to be representative (Groves, 2006).

The second is who responds. Some demographic groups are more likely to complete mail and telephone surveys. For instance, older residents return postal surveys at higher rates than younger ones (Li et al., 2026). Differences in who completes a survey can lead to different survey responses. This is called nonresponse error in survey science. Specifically, when some residents are more likely to participate in a survey, the data produced by this survey may not represent the community’s perspectives (Groves, 2006). 

The usual answer to these issues is statistical modeling such as weighting. Weighting counts under-represented groups’ responses more heavily to stretch the data to look like the community. Weighting is legitimate and surveys should use weighting, but it can only stretch the voices that showed up. It cannot create the ones that didn’t. For instance, if a city’s population is one quarter young people and only a dozen respond, weighting makes those dozen speak for a quarter of the city. The results look good until someone checks how many respondents the survey actually counted.

What the Modern Standard Looks Like

Though the math behind traditional surveys is still sound, the challenges these surveys are now facing are making room for change. A traditional survey starts with a list and hopes the people who respond to it add up to the community. A modern survey starts with the community itself. Census data provides the same targets as traditional surveys, but a modern survey goes out to meet residents where they are.

Nearly all American adults are online and, at Zencity, we reach people inside the apps and sites where they spend their time (Pew Research Center, 2025). We recruit broadly across many channels so that anyone in the community has a chance to share their voice. This approach finds a very different group than a link posted on the city’s own channels, which only ever reaches residents who are already engaged. 

We also use other approaches to reduce error. For example, our survey invitations don’t announce the topic up front so that they bring in all sorts of people, not those who hold a strong opinion about it. That’s precisely the group a public hearing never hears from. 

Finally, because online surveys are distributed in real time, we can monitor who is responding, and adjust recruitment to focus on whichever groups are falling behind. The sample includes representative voices from the community to start with, instead of hoping that statistical correction after the fact can do it. This leaves weighting doing the job it is good at, closing a gap of a few points instead of standing in for residents who never appeared.

Built this way, modern surveys can produce representative data in several weeks, rather than several months, at a much lower cost. Across a budget cycle, this is the difference between an input into the decision and a record of what residents thought after it was made. 

Local government work is too consequential to rest on whoever happens to answer the phone, and hearing from a whole community is not a matter of effort or good intent, both of which every government team already brings. It is a matter of methodology.

——————

Zencity supports 400+ organizations in hearing from more than 200M+ residents, understanding community priorities, and taking confident, data-backed action. Built to fit into the teams and workflows local governments already have, the platform supports planning, everyday communication, and decisions that are supported by defensible data.

Let’s talk about bringing this methodology to your community. Request a demo with the Zencity team.

Co-written by Ale Aponte, Orly Davidov, and Michael Rosenbaum at Zencity.

References

Blumberg, S. J., & Luke, J. V. (2025). Wireless substitution: Early release of estimates from the National Health Interview Survey, July–December 2024. National Center for Health Statistics. https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless202506.pdf

Einstein, K. L., Palmer, M., & Glick, D. M. (2019). Who participates in local government? Evidence from meeting minutes. Perspectives on Politics, 17(1), 28–46. https://doi.org/10.1017/S153759271800213X

Einstein, K. L., Glick, D., Godinez Puig, L., & Palmer, M. (2023). Still muted: The limited participatory democracy of zoom public meetings. Urban Affairs Review, 59(4), 1279-1291. https://doi.org/10.1177/10780874211070494

Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675. https://doi.org/10.1093/poq/nfl033

Kennedy, C., & Hartig, H. (2019). Response rates in telephone surveys have resumed their decline. Pew Research Center. https://www.pewresearch.org/short-reads/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/

Li, X., Ying, N., Li, K. Q., Shi, X., & Miao, W. (2026). Correcting nonignorable nonresponse bias in turnout estimation using callback data. Political Analysis. Advance online publication. https://doi.org/10.1017/pan.2026.10035

Meyer, Bruce D., Wallace K. C. Mok, and James X. Sullivan. 2015. Household Surveys in Crisis. Journal of Economic Perspectives 29 (4): 199–226. https://doi.org/10.1257/jep.29.4.199

Martin, O., & Venugopal, A. (2026). Participation and representation in local government speech (arXiv:2604.21202). arXiv. https://arxiv.org/abs/2604.21202

Pew Research Center. (2025). Internet, broadband fact sheet. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/

Related Posts

phone survey blog
Articles
Michael Rosenbaum

The Hidden Problem With Traditional Community Surveys

Public safety isn’t just about crime rates, it’s about how safe people feel. Together with the National Policing Institute, Zencity has launched the Public Safety Confidence Index: the first nationwide measure of Americans’ trust in law enforcement and their sense of safety. Built from over 20,000 resident voices, the Index offers leaders and communities a new data-driven way to listen, build trust, and take action.

Read More

Related Posts

phone survey blog
Articles
Michael Rosenbaum

The Hidden Problem With Traditional Community Surveys

Public safety isn’t just about crime rates, it’s about how safe people feel. Together with the National Policing Institute, Zencity has launched the Public Safety Confidence Index: the first nationwide measure of Americans’ trust in law enforcement and their sense of safety. Built from over 20,000 resident voices, the Index offers leaders and communities a new data-driven way to listen, build trust, and take action.

Read More

Ready to get started? Reach out to our team to schedule a demo.