By Dr. Adam Probolsky, President of Probolsky Research and senior research fellow with Claremont

You want to understand what your community knows, feels, and thinks, but hiring a professional polling firm is out of reach at the moment. I am going to violate the pollster’s code and share the secrets of how you can fill in the gaps until you can convince your policymakers to pay for the real thing.

Disclaimer: I am not being gratuitous in saying that there is no substitute for statistically valid research. And it would be research malpractice for me to suggest that you should enter into any kind of serious situation without all the right professionals involved – pollster + outreach consultant when considering a tax or bond, pollster + PR consultant when a crisis hits, pollster + lobbyist when pursuing legislation.

Now, here is the pollster’s equivalent of the magician showing you how he saws the woman in half.

1. Grab your favorite management analyst. You know the one that everyone goes to for help with tech stuff, Canva, and to explain the latest social media trends, “Jackie, WTF does 6-7 mean!” She will be your best friend throughout this process. If you are that person in your organization, you are already ahead of the game.
2. Decide on your goal: Is this a general exercise to capture the “how are we doing” information? Or is this focused on [fill in the department or policy area here]? Keeping the questions on topic can be a challenge – be the gatekeeper.
3. Now you get to reverse engineer the questions you are going to ask. Hint: Pollsters don’t ask clients what questions they want to ask. We ask what information they want to know. Most people are really bad at writing survey questions. So we start with having staff complete these sentences as many times as they want: “I think that…” and “I wish I knew…” 

Example: I think that everyone is mad about the 25% increase in water rates. 

→ Q: Thinking about water rates compared to the value you receive, would you say that you pay too much, too little, or just the right amount?

Example: I wish I knew if every man, woman, and child in the city is as obsessed with pickleball as the loudest voices suggest they are.

→ Q: How much of a priority is it for you and other members of your household that the city invests in new pickleball courts?

If you are brave enough, send an email to department heads and ask them to tell you all their ‘I think that’, and ‘I wish I knew’ ideas. Cull through them and use the best ones.

1. Protect your residents from survey fatigue. You would be surprised at how long they will engage with a survey being conducted by their local government agency. Ten to fifteen minutes (20 – 30 questions) is a perfectly OK place to shoot for. But remember, a question about the new community center or wastewater treatment plant that would take you 20 seconds to read and answer might take them a couple of minutes. They do not live and breathe public policy, and they certainly don’t ever think about full tertiary treatment of effluent. So give them the space and time to digest what you are asking and to answer substantively. 

1. Set internal expectations for participation and the value of the data you will collect. Posting a survey on your agency’s website and promoting it on your social channels is unlikely to yield the volume of responses you want. Similarly, the results from this unscientific effort will in no way reflect the broad opinions of your residents/ratepayers/voters. I say this without the intent of puffery for pollsters. I am just pointing out that the opt-in nature of a survey you promote, or even the ones conducted by companies that offer subscriptions for “quick polls,” will be inherently unbalanced. You do not have the tools (or budget) to match the demographics and geographic breaks of your constituency.

But that is not a reason to throw up your hands. The real value comes from residents seeing that you cared enough to ask what they think.

1. Whatever self-service survey platform you use – SurveyMonkey, Google Forms, PublicInput – will give you a somewhat visually pleasing set of topline results. Creating other reports can be tricky – this is where your management analyst friend can come in handy. The best part of your in-house survey results will be the open-ended comments to your questions. There will be a lot of hate and ignorance on display. But if you can see past that, you will learn new things about the community you serve.

Caution: If you publish the results or even just share them beyond staff, you are going to open yourself and the agency up for criticism. The respondents will be whiter, older, and richer than occurs naturally. Professionally, we call this data “interesting,” “directional, or “valuable for discovery.”

Return to Communicator