Lots of people want to have mental models tomorrow, without all the hard work that goes into collecting the data and massaging it until it takes shape. I hardly blame them. When someone asks me how this can be done, I encourage them to sketch a mental model together with their team, over the course of two hours. This model can be a representation of what they know now about peoples' behaviors and feelings and philosophies, written from the perspective of these people, with pronouns like "I" and "My." It's when someone says, "Can't I just send out a survey and collect the data?" that I start to shake.

Yes, I shake.

Surveys are a cop-out for this kind of work. Surveys don't let you have conversations. Surveys don't give participants the opportunity to tell you their perspective in their own words, in the order they think is important. Most of all, surveys don't let participants tell you things you wouldn't think to ask. All of these statements are patently true about surveys. (Tell your boss.)

Yet for some reason folks respect surveys and the data gathered thereby. Somehow, as a population, we believe that the more people we ask a question of, the more significant the result.

The Webster's definition of "survey" (the noun) is "a broad treatment of a subject" or "a poll." A Wordnik definition more specifically states "a gathering of a sample of data or opinions considered to be representative of a whole." Look at the words poll, opinions, and data. The last word is the one organizations tend to set more store by. It is a word that inspires confidence and reduces risk simply by sounding scientific. Yet, truly, most surveys you have encountered at your organization are about opinion. "How did you rate our service today?" Or, a more subtle example: "Which of the following brands of chocolate do you typically buy?" (What if the brand you just bought isn't listed? What if you don't buy chocolate? What if the majority of the population the organization supports doesn't buy chocolate, yet the organization get duped into thinking that 63% of people buy Ghirardelli?) The surveys about data have to do with, for example, assessing biodiversity on a particular island, not with people or what they are conjecturing. So, if surveys that we normally see have to do with opinion, preference, and conjecture, what is actually "scientific" about that? We know opinions and preferences change over time. They change based on how the question is asked. They change based on who is listening. We know that conjectures are just guesses about future reactions and behaviors. And surveys about past activities rely on imperfect or biased memories and recall. How can opinion and conjecture inspire the confidence we need for design?

There are other things we know about surveys. They can be written purposely or by accident to reach a certain conclusion. The result can be greatly influenced by whom you ask to participate. They can grow long and so tedious that participants start marking answers randomly. No matter how deeply we think when we write the answers to a particular survey question, it is still a bounded list. Participants have to match the answer they would have said out loud to one of the things we thought to write down as a selection--if anything is even close. Participants feel compelled to select an answer even if none of the possibilities is close.

If we peer at what we frequently use surveys for, if we really break it down to its root, it is to persuade or provide evidence to support a decision. Or its use is to get attention, as in the news media. These are valid uses of surveys; I am not judging, here. What I want to point out that what we do, "design," is not "persuade" or "decide" or "get attention." We are doing something that is different than marketing departments or executives seeking to change the perception of a product/service on the market. "Design" is closer to "invent," "devise," "formulate," "plan," "create," or "contrive." We cannot use surveys to support our design activity.