Posted by: rolfsky | August 19, 2008

sampling myopia: do you know when to ask?

While walking home the other day, I made an observation which I thought I’d share in a very Seth Godin way. Here goes:

I walk to and from work, usually waving hello and exchanging a few words with a neighbor who seems to incessantly prune her shrubbery. For a few days now, her son has been home from university, and each time I’ve seen him out in front of the house, he’s been working on his mountain bike(s).

After seeing him work on his mountain bikes a few days in a row, I thought to myself:

“man, he must really like mountain biking! every time I see him he’s working on his bikes!”

For a brief second I considered saying something like that to him, but then thought about how he might respond. Because we’d only encountered each other as I pass by my neighbor’s house as I walk to and from my work, he could have easily said,

“man, you must really like walking! every time I see you, you’re walking to work!”

On the surface, this might seem like a silly interaction, but what it uncovers is a deeper problem when attempting to do any product research or marketing. Based on my experience with my neighbor’s son, I would have attempted to market the “extreme sports” angle of my product, but that might be completely mis-directed.

Perhaps he was working bicycles so much because he was planning on riding a “Race for the Cure”, and his real interests are in volunteer work. Or maybe he was repairing bikes for a friend. Or any number of alternate possibilities.

However, if we only construct our customer personas from questionaires we send them in the mail, we are likely missing broad swaths of their personality, goals, and aspirations.

Aggregated action-based data (search data, credit card receipts) can help alleviate this single-sampling problem, but then you run the risk of missing the context, like an Amazon customer who only buys products because she can have them gift-shipped to her nephew.



  1. Rolf, I am guessing you are hitting on the constraints of machines here. There is simply no way yet to be able to interpret data that relates to human behavior. Even we humans suck at it when interacting with each other. But we still do better then any algorithm.
    I wrote a post about this phenomenon earlier in which I questioned the value of mining our on-line actions and interactions. I gave this example:

    You are sitting with a friend and he tells you enthusiastically about a movie he has been too. Makes you want to go yourself, right? Why does that work? Because a lot of things happen at the same time. In the physical world there are many different stimuli that affect your behavior. Things like speech, sight, hearing, touch, feeling, movement, trust, relationships, common experiences or taste, context (like the fact that you are hanging out at your home together), all come together in your brain providing you with a feeling of value to your friends story. It also provides you the opportunity to DISAGREE with your friend.

    But in the on-line world you lose most of these stimuli. In Facebook you get flattened stimuli from the newsfeed: “Alex went to this movie and he liked it”, or personalized ads using the profile information. But there is no “hanging out together, no voice, taste, touch or other stimuli, no way of agreeing or disagreeing with your friend.

    I bet that if you would have walked over and had a conversation with him, you would understand why he was working on his bike every day. Asking is so much easier than inferring 😉

  2. […] 21, 2008 · No Comments Rolf Skyberg just wrote a “Seth Godin”-like post (his words)  I like a lot. He describes why it is so difficult to understand customers on the web, […]

  3. As a researcher I agree 100%. That’s why I’m such a big fan of research triangulation. Questionnaires are important. Analytics are important. But the 3rd leg of the stool (corner of the triangle? I’m getting lost in my metaphors here…) is ethnographic research or contextual inquiries – where you actually go sit with your neighbor in his garage, talk to him, observe him, and understand what he’s doing to his bike and why. Companies really need to get out more and go see how their customers use their products in the real world…

  4. […] recently posted about “sampling myopia” — the idea that it’s unlikely you’ll get a “good” answer from […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: