Benchmarking the Community Survey and Blockwise

We aim to help Zencity clients understand their results and put them into context. To that end, we compare each client’s results to the United States as a whole and against communities with similar characteristics such as size, population density, and region (their “cohort”). These cohorts help to distinguish patterns unique to each client from those that are common in similar communities. (For example, urban communities might score higher on average than suburban or rural communities on questions about public transportation, but score lower on questions about affordable housing.) 

To generate national and cohort scores, we conduct a national survey with the same content as our recurring surveys. The Blockwise National Benchmark Survey is conducted quarterly, while the Community Survey National Benchmark Survey is conducted semi-annually. 

Figure 1 shows an example of benchmarking in the Community Survey report. The blue dot represents the client’s score, the square represents the cohort score, and the triangle represents the US as a whole. 

As shown, this particular client is a bit lower than the US and the cohort score for availability of affordable housing and for the sense of overall safety, but it is substantially higher than both US and the cohort when it comes to the availability of a variety of art and cultural events. 

Figure 1. Benchmarking the Community Survey. 

In this article, we describe the methods of creating cohorts and generating scores. 

Creating the cohorts

First, all cities, towns, and counties in the US--not just Zencity clients--are put into cohorts based on geography and demographics using a method known as cluster analysis. This method balances a range of different characteristics to put communities into cohorts that are statistically similar to one another. The result of this step is one set of cohorts for cities and towns and a second set for counties.

For example, one of our city cohorts consists of 480 cities. This group is made up primarily of small cities with populations of less than 100,000 residents, mostly in the northeast. They are characterized by high population densities, large numbers of young residents, and high levels of racial and ethnic diversity. In addition, a larger proportion of residents are immigrants compared to the rest of the United States.

Another city cohort consists of about 10,000 suburban and rural communities in the Midwest, almost all below 50,000 residents. These communities tend to have low population density, and residents are, on average, older and more likely to be White than the rest of the country. 

We chose this method to create objectively similar groups of communities with respect to characteristics that are likely to be correlated with answers to the survey questions. 

By creating cohorts of larger sets of communities we have found that our benchmark baselines tend to be more reliable than they would be if we had direct comparisons of a small number of communities.

Generating cohort scores and national benchmark scores 

Information on scoring for clients are found here. 

The cohort scores are the weighted average of scores for respondents in each cohort. These weights are calculated to match the client’s own demographics, to make them directly comparable to the clients’ scores. 

Because we are creating a cohort from the national benchmark survey data instead of running representative surveys of selected communities, most cohort scores are generated from a few responses from each community. 

We can share the names of the communities in the cohort so you can get the sense of who's in your cohort, but because we average across communities (as opposed to getting a representative sample of City A, B, etc), you will not be able to compare results against these individual communities.

We calculate the national scores in a similar manner. The national scores are the weighted average of scores for respondents across the national benchmark survey. These weights are also calculated to match the client’s demographics.