User>Driven

Last week I attended a webinar on developing personas put together by User Interface Engineering. Jared Spool was a bit distractible but explained things in a very approachable way. He described IUE's research methodology for creating personas, which involves 6-12 site visits, extensive note taking, analysis, persona development and incorporation of the personas into the development process.

Here are a couple of take-aways I thought you might find interesting.

Personas Are Not Fiction

A while back I wrote a piece about personas where I talked it about why it's a bad idea to base your personas on a single real individual. Using a real person as a substitute persona might seem like a good idea since you know that everything about that user is for real. The problem, though, is that one individual comes with quirks that may not be representative of the market as a whole. If you interview enough people, you can eliminate their biographical peculiarities by focusing your personas on the characteristics common to the group.

Jared also cautioned against adding a lot of biographical detail to personas, but he came at it from the opposite direction. He warned us to watch out for the temptation to make up background info on your personas. He did recommend giving your persona a name (though not the name of a real research subject for the reasons above) and even finding a reasoanble-seeming photo of that person from stock photography. Both of these are to make the persona more real, easier to remember, and easier to talk about. He said, though, that the color of their eyes, the kind of snacks they like or other details that don't relate to the product you are trying to design are just distractions. He argued that you should eliminate any detail from the persona that you can't connect with a design decision you need to make.

I asked about his inclusion of age as a detail in an example persona they were showing for a medical application and he reasonably replied that age, if representative of the real people you interviewed, could tell you something about the level of computer-sophistication of the user. He emphasized the word "representative," saying that otherwise you were "just making stuff up," and that didn't make for useful design tools.

See a great response Jared posted on his blog to a rant by Jason Fried of 37Signals about personas being "artificial, abstract, and fictitious." Jason sure missed the mark here and I was glad to see so many responders taking him to task.

Clusters Lead to Personas

Another thing I took away from the seminar was a method of extracting data from interviews and turning it into personas. I usually take a lot of notes during customer interviews. When I am finished with them all, I go through and look for key data points like reports people want to see, goals they have, frustrations they have, tools they use, etc. Then I count up how many people use each tool, for example, to get a picture of what people as a whole are using. This is not a scientific, quantitative approach, but it's often directionally helpful.

The folks at UIE have developed a method of clustering these data points about their subjects (they call them "informants" but that's a little too CIA for me) that helps them develop multiple personas. They compare and contrast the individuals they interviewed and plot each on a series of sliders. They pick a subject they interviewed them on, say, computer-savviness, and they plot all the users relative to each other. They do this for all aspects they can then look for patterns in the charts where individuals seem to be close together on several sliders. Looking at what these clusters of neighbors have in common is how you then generate the descriptions of personas.

One of the key things this method does is to weed out irrelevant detail. It might be that all of the neighbors in a particular cluster are young while other clusters seem to skew older, so that's a good bet for a detail to be included in the persona. It might also be that the majority of your subjects overall drive Fords but if you find there is no consistency as to automotive brand choice within clusters or little contrast across clusters, then auto brand preference is probably not relevant to the personas you are going to write up.

I find I like the rigor this clustering concept seems to bring to the qualitative process of developing personas and I plan to try it out on my next project. Hope it's useful for others as well. 

In an interview with User Interface Engineering, Kim Goodwin of Cooper talks about her company's approach to interaction design using personas and what she calls "scenarios" (what I would call "use cases.") She calls their methodology "Goal-Directed Design" because it focuses on the goals of the user and of the organization providing the product or the service. (She points out rightly that "If design teams only focus on the persona's goals to the exclusion of making profits, the product won't be successful.")

One other point she made caught my eye because it comes up often when creating and discussing personas. When asked if you should base your personas on real people, she said:

Certainly there are some real people who are very similar to a persona the design team may create, but it's a dangerous approach because real humans are idiosyncratic. For example, any individual user might hate the color blue or have some other random opinions that aren't necessarily representative of a larger population.

One of the strength of personas is that they gloss over those little idiosyncratic things and really focus on the essence of what is common to this particular type of person. That's one of the reasons why we rely on personas instead of real users--they are more representative.

People like to make personas as real as they can and impart personality and preferences to them. We give them names, for example, which I think helps to make them real to developers. I've been in discussions of the brand of coffee a software developer persona would prefer, though, and I think that's taking it a bit too far. It's important to talk to enough real users that they begin to blend together (like fine coffee) into a composite that's useful and not overly specific.

This is qualitative research, though, not quantitative, so (as Kim also points out), that doesn't mean you have to take months or spend millions on research. I like to do 10-15 telephone interviews to begin to understand and characterize the personas for a particular project. The clue that you haven't done enough interviews is if you keep referring to a single real life customer by name when talking about project decisions.

The key aspects of a persona are their goals and behaviors. Goals are things like making an airline reservation, getting their printer to work again or meeting a client deadline. Behaviors are things like getting impatient after waiting for a page to load for more than 10 seconds, refusing to use Internet Explorer or doing online research and then buying at the store.

These goals and behaviors are abstracted from individual people so that the individual quirks of those people fall away. Go ahead and add names for these people that help you imagine them and refer to them in documents and discussions, but make sure you don't distract yourself with the names of any actual subjects.

Ever had the feeling that no one actually reads the PRD, personas and use cases you carefully and lovingly craft from the many months of conversation you've had with customers? Ever wished you had more time to read similar documents created by your peers?

Kevin Cheng of kev/null may have an answer for us in the form of comics. Kevin is a senior interaction designer at Yahoo! where he and his colleagues are creating simple, expressive comics to quickly communicate the essential ideas behind features. In a recent interview with Ashlee McKee at UIE, he says a major reason is that they discovered in many projects that "the documentation would either not be read thoroughly enough or each person would interpret the meaning of the requirements differently. Two months later, we were deep in the development process and suddenly realized everyone had different visions of what the requirements actually meant."

He argues that simple comics of 5-8 panels showing how a proposed new feature would be used are easier to create, easier to understand, and more likely to be read than formal use cases. The key is to keep it simple and to concentrate on the user rather than on the interface. Check out Kevin's presentations filled with fun examples of how to put this idea to work.

Jacob Nielsen published an excellent article way back in 2000 suggesting as few as 5 users were enough to adequately test a design. He shows data suggesting you gain 85% of the information it's possible to gain in user testing of a single design with 5 users and that more users provide less and less unique feedback. After that, he says, you might as well revise the design and start with 5 new users. Revise one more time and test with 5 more and you probably have as good a design as you are going to get.

I liked this article not just for it's pragmatic approach to research but also because I think it shows how easy and approachable doing product research really is. Companies often shy away from market and user research, I think, because it sounds intimidating. You need to sponsor customer advisory boards, organize focus groups, compose elaborate questionnaires, and so on, it seems. That sounds hard, takes a lot of time and money and requires expertise. Well, I'll agree with one of those.

In fact, you can learn a lot by just putting your designs in front of a handful of customers and asking them to try them out. You can also call up a half dozen customers and ask them about their biggest frustrations (within your are of expertise) and get a lot of information to feed into your feature prioritization.

I do think you need some expertise to do this right, however. As I have argued recently, someone in the organization must be dedicated to listening to the market - to customers, potential customers and users - to find out what the opportunities are for solving problems and whether proposed solutions are likely to solve them. These people can't be salespeople or sales support people. They will usually be product managers and design or usability professionals.

Once you've invested in hiring these people, though, you don't usually need an enormous research budget to support them. You can get the qualitative input you need with just a few customer interactions. (Though quantitative market sizing data may take a little more effort - see my recent entry on this.)

Is 5 subject enough in your experience? What's the fewest number of subjects you've used and felt like you met your goals? What is the largest number you ever used and why? Post your comments below. 

How many times have you given up on taking a survey because none of the multiple choice answers provided fit your situation or because the questions themselves were based on assumptions that didn't apply to you or because they failed to ask the questions that were most important to you? And why do some surveys seem to go on and on, asking question after question to the point that you want to quit?

The answer is that many surveys are misused. They are used as all-purpose feedback tools - and often they are the only feedback tool a company uses. A lot of companies do an annual customer satisfaction survey. (Some also do an employee satisfaction survey.) They look at this as their one and only chance per year to get good customer input so they load the survey with every question every department can think of.

And no question can ever be dropped or changed from the annual survey, of course. That would break the chain of comparability between results of different years. Participation drops every year, but we can make that up with incentives, reminders, and by sending it out to pester ever more customers each year.

Nobody knows what to make of most of the data, but we get an overall customer satisfaction score that hovers in the 80s every year and that sounds pretty good so we must be doing okay.

These familiar situations arise when well-intentioned people try to use a quantitative tool like a survey before they've gathered enough qualitative information. The surveys don't work because the people designing them fundamentally don't know what to ask.

Quantitative Research

Quantitative research involves measuring things. The tools used for this include surveys, web analytics, data mining, modeling statistical analysis and predictive analytics. Concrete things like conversion rates, revenue, lift, demographics and even customer satisfaction can be reduced to numbers and compared to industry norms, past metrics, goals, etc.

Quantitative methods are perfect for getting at the nuts-and-bolts what, where, when and how many questions needed to measure your progress against your goals, competitors or standards. Hence the annual customer satisfaction survey. There are even statistical methods that can help you prioritize features based on the value different customer segments place on them and how much they change their likelihood to buy. Sounds good, right?

But how do you decide what your goals are in the first place? What good does it do you to measure something if you don't know what to measure, why you are measuring it, or what to do with the resulting numbers? And how do you know the list of features you want your customer survey to help you prioritize has the killer features your customers really want in it? And worst, what do you do with this nagging feeling you are missing some critical insights into how your customers think?

Qualitative Research

Qualitative research is all about exploring. The tools used for this include focus groups, interviews, on-site shadowing, and even playing games with your customers. The result is not numbers but lists of insights about customer needs, goals, features, obstacles and values, as well as descriptions of personas, tasks, processes, relationships, mental models and value propositions.

Qualitative methods get at the why and how questions you need answers to before you can formulate the right quantitative questions. The most familiar example I can think of from the world of product management is feature prioritization. Doing a number of interviews of focus groups can help you generate a list of feature ideas that respond directly to the problems you see your customers having in the real world.

This kind of exploratory research where you are trying to learn about your customers' lives or businesses and hearing about or observing their problems first hand is where you discover needs you never knew about and where really innovative product and feature ideas are born. You might find out they don't need more features at all but better documentation or training or support or whatever. You might find out they think your product is worth way more than you charge. You might find out they couldn't care less about the new product idea you are about to spend a year developing. Without the kind of insight that comes from getting to know individual customers and how they think you can't find the hidden nuggets, the ideas that will really make your product a must-have.

Okay, now you've done the interviews or focus groups or onsite visits you needed to do to really put yourself in your customers' shoes. You've learned all about how they think and you have a lot of ideas about products or features that could solve the  most painful problems they face every day. How do you know which of these brilliant new insights are the most valuable, the ones that will drive people to buy, the ones you should implement first?

And Back Again 

That's where the survey comes in. Once you have the proper list (of features, in this example), you can do your survey and quantify how many potential customers in which segments are motivated by which of these features. Good statistical analysis of a properly executed survey can help you size the market for different products or features. You might find, for example, that feature X is valued by nearly everyone in your target market but valued below some other features that are different for different groups. That might suggest you need different product versions or add-ons for each group.

This qualitative before quantitative approach works with the annual customer satisfaction survey as well. Before you do your first survey, bring some customers in for informal discussions to explore their priorities and needs. Then you'll be able to develop a survey that measures whether you are satisfying their needs and not just your need to brag about customer sat numbers. 

Innovation Games

I attended a seminar last night sponsored by the Boston Product Management Association (BPMA) on a an interesting (and fun) method of qualitative research called Innovation Games. Mara Krieps of Pivotal Product Management walked us through a dozen different games you can play with your customers to learn more about them, their values and their need, and get valuable feedback on your product.

Drop by the PM Exchange Forum to learn more and share thoughts about games you can play with your customers.