How to measure the impact of community engagement
Have you ever asked someone what makes their neighborhood feel like home? Why they stood in line for the perfect burrito? Or how the perfect commute can affect their work day. Urban design is fundamentally about understanding how people live and work and create together. That understanding is born out of learning to ask the right questions, and measuring the impact of your designs.
Human-centered design is not only about respecting cultural and social nuance, but incorporating the voices of participants at every stage of design. To accomplish this, you need a solid framework for design research, the ability to procure accurate data, and the storytelling chops to weave that data into design.
What do we mean when we talk about data?
When you hear the word ‘data’ what is the first thing that pops into your mind? The term generally conjures up images of spreadsheets, or rows of numbers. This is only one type of data though, called quantitative data. The term ‘quantitative’ refers to that which can be counted, such as the number of steps we take per day, or our average heart rate. Qualitative data on the other hand, is usually descriptive, such as how hard we feel we’re working when we get to the tenth mile of a marathon.
Both types of data are useful in identifying and understanding our actions, thoughts, and emotions.
Crafting a user journey
After a quick teaching session explaining various forms of personal data (measuring actions, thoughts, and emotions) we asked participants to develop a user journey for visiting a street fair. This included writing out individual story beats on post-its and organizing them along a timeline. We then had participants identify various actions that visitors could take at each ‘beat’ of the journey, such as visiting a food truck, ordering a taco, sitting and listening to music, or people watching. This was followed by participants then identifying various thoughts and emotions visitors to the street fair might have, like was it worth it to stand in line? Did visitor to the fair feel as if they were part of a community?
Creative ways to collect data
After we had established a user journey, and identified points of actions, emotions, and behaviors along that journey, we had participants brainstorm creative ways to measure interesting points of data. For example, one participant suggested having a trash can for delicious food, and one for not-so-delicious food. At the end of the event, which was more full?
We emphasized that participants should think about low-tech and low-cost methods for collecting data. This serves two purposes—to break down the concept that data collection requires extensive expertise and to demonstrate that impact measurement can be woven into any kind of project or event.
What about social media?
Have you ever taken a photo of your food for Instagram? Food photos actually tell us a lot about the kind of culture we are. One thing they don’t show though, is how the food actually made us feel. You can’t take a photo of regretting that you ate a burrito. Too often we rely on number of likes, or retweets as a metric for the success of community initiatives. While social media can be useful for understanding basic quantitative information, it can be difficult to find useful qualitative data.
Surveys are a standard way to collect data , but these are usually given after the event is over. By incorporating impact measurement throughout the course of an event, you’re more likely to elicit unbiased responses from visitors.
Some of our results
Ultimately the goal of the workshop was to introduce participants to the concepts of using low and no-tech data, to tell a story about place, and to measure the impact of potential community initiatives. Here were some of our results:
How many people are arriving by bike? When people show up, give them a token for 10% off at any vendor. At the end of the event, count the number of tokens that were passed out and then collected (the difference will also show how engaged visitors are with the booths).
Was the burrito worth it? Set up two trash cans on either side of the dining area. Over one, put a sign that says “Loved you food? Throw your trash out here.” Over the other, write something like, “Something missing? Throw your trash out here.” Gauge which trash can has more volume throughout the event.
Did you miss out on shopping at a booth because it was too crowded? Hang up a book of “rain checks” on the outside of each vendor booth. Visitors can grab a rain check if they wanted to spend time at the booth, but we’re unable to. Subtract the number left in the book from the original number.
How much were people shopping? Place a scale near the exit, and encourage people to measure their bags on the way out.
How you can do this yourself?
In addition to sharing the results of the workshop, one of our main goals was to create a framework for anyone to use. In summary, our method for creating impact measurement has four parts:
1) Create a user journey for your community activity
2) Think through the possible data that can be collected at each stage of the journey
3) Determine what data points will help you tell the best story of the activity
4) Get creative and come up with low-cost ways to measure
This workshop was made possible through LearnDoShare NYC. This year’s theme was DIY urbanism, and the conference explored “how collaborative action, design thinking, storytelling, play and technology can be used as tools for civic engagement and social good.”
I was pleased to have worked with LearnDoShare, and to continue to work with the conference throughout the year.