Every day, city budget decisions, council votes, and resident communications rely on having accurate survey data on their community. Surveys are collected in a council chamber, via comment boxes, or by mail are common sources of data. Too often, these sources do not provide insight that represents the whole community because they do not gather data from a group of residents whose characteristics are similar to those of their community.
What does representative mean?
Representative doesn’t mean a big sample or a survey that “reached a lot of people.” Rather, representation is about reaching the “right people.” Data are representative when, as a whole, the people who answered a survey are similar to their community. In the context of local government, “the community” is often the residents that a decision is supposed to serve. This resemblance doesn’t happen by chance; it’s design. Representative surveys are built using a specific approach, where every step is carefully checked and adjusted when needed.
How does Zencity produce representative survey data?
Here are the six steps our methodologists use to ensure that the data produced by Zencity is representative, and to know that clients receiving data their team can stand behind.
- Start with the community. Before any surveys go out, we define who we want to get responses from. This means determining who does this sample need to represent and in what dimensions? Survey scientists often consider age and geography when defining the community, but other characteristics may also be important to decision-makers. For instance, whether residents are renters or homeowners is relevant for housing policy. Other characteristics, such as income, may influence how a resident answers questions on budgeting priorities. We use estimates of what share of the community they account for to set recruitment targets for the survey sample. These targets come from real, high-quality data. Specifically, we set them using data from the Census and American Community Survey, which the Census Bureau updates on a rolling annual basis so local population estimates don’t go stale between decennial counts (U.S. Census Bureau, 2025). This step is important as skipping it makes it impossible to be sure of who a survey is representing and if the sample does this well.
- Size the sample. A large sample is not the same as a representative sample. A sample is representative if it gets enough responses from every group that it sets targets for, including hard-to-reach groups. Considering groups when setting sample size is important as a citywide sample of 800 can still be sufficient for generating representative data on the city, but too small to accurately capture the perceptions of a group that is important for a decision (such as a district where a policy will be implemented). Because of this, we size the sample to represent important subgroups, not just the community as a whole. Not doing this could mean our hardest-to-reach group is left out of the sample.
- Distribute the survey across many channels. People who are hard-to-reach by mail or landline are often easy-to-reach on the web, an Instagram feed, in a mobile app like Candy Crush, or in languages other than English. We recruit across many channels at once because reaching people where they are (online) simplifies the task of reaching the targets set for representative surveys. Effectively, this approach closes the gap between the residents a city hears from by default and those it has to go find.
- Make participation easy. Residents are juggling multiple priorities and this makes it hard to respond to a 30 minute, 117 question survey, even if they wish to participate. We try to reduce barriers to participation by conducting surveys that are less than five minutes long, natively displayed on an iPhone, are accessible to a screen reader, and can be completed anywhere, including during a morning commute. Reducing barriers to participation also supports us in getting representative data by making it easier to meet survey’s sample targets.
- Protect data quality. Every open online survey attracts a share of bots, speeders, and duplicate submissions, and the methods literature on online data collection has been documenting the scale of the problem for several years now (Storozuk, Ashley, Delage & Maloney, 2020). Straight-lining, impossibly fast completion times, and repeat submissions from the same device all have to be caught and removed before anything else happens to the data. At Zencity, we designed automated checks that help us detect this behavior and drop any observations that do not meet our quality standards. This step ensures that we do not keep fraudulent or poor observations that threaten representativeness by providing inaccurate information on the community.
- Weight. Check. Finalize. Careful recruitment alone rarely produces a sample that perfectly meets its targets. We use a technique called weighting, which makes statistical adjustments to the final dataset so that our final sample matches the targets set for the sample (Deming & Stephan, 1940; Battaglia, Hoaglin & Frankel, 2009). For instance, if a group needed for a target sample of 100 comes in at 95 responses, we weight these responses so each counts for roughly 1.053 responses. Therefore, the results count that group as the right proportion of the population. This is not inventing five responses. It’s making sure the 95 who did answer count for what they represent. Ideally, weighting should result in small adjustments. This happens when the previous steps work as expected.
Each of these six steps carries some weight, but they don’t make representative data on their own. Skip step one and steps three through six and you cannot ensure the sample will have the same characteristics as the community. Skip step five and the sample could contain noise instead of signal. Skip step one and a group that is very important for decision-making is the one too small to be useful or too small to weigh well (i.e., with small weights). The order is the method, which is exactly why “representative” is made by design and is a claim that can be checked at every stage.
That’s what the result looks like from the other side of the table: data local leaders can actually stand behind, because they, or anyone reporting to them, could walk through how it was built, step by step, and find nothing hidden along the way. Representative input, arrived at through a process like this one, is what lets a decision get made and defended with confidence.
——————
Zencity supports 400+ organizations in hearing from more than 200M+ residents, understanding community priorities, and taking confident, data-backed action. Built to fit into the teams and workflows local governments already have, the platform supports planning, everyday communication, and decisions that are supported by defensible data.
Let’s talk about bringing this methodology to your community. Request a demo with the Zencity team.
Co-written by Ale Aponte and Michael Rosenbaum at Zencity.
References
U.S. Census Bureau (2025). American Community Survey (ACS). https://www.census.gov/programs-surveys/acs.html
Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got Bots? Practical Recommendations to Protect Online Survey Data from Bot Attacks. The Quantitative Methods for Psychology, 16(5), 472-481. https://www.tqmp.org/RegularArticles/vol16-5/p472/p472.pdf
Deming, W. E., & Stephan, F. F. (1940). On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals Are Known. Annals of Mathematical Statistics, 11(4), 427-444. https://projecteuclid.org/euclid.aoms/1177731829
Battaglia, M. P., Hoaglin, D. C., & Frankel, M. R. (2009). Practical Considerations in Raking Survey Data. Survey Practice, 2(5). https://www.surveypractice.org/article/2953-practical-considerations-in-raking-survey-data
