It’s hard to uncover behavioral qualities during a focus group session, so the technique suffers from “what I say isn’t what I do” syndrome.
Focus groups have been a key method for marketing and market research departments for so long that they seem to have become incorporated into the product development cycle. Get an idea, make some mock ups, run some groups, receive enthusiastic responses, build the product, then wonder why it didn’t do as well as you expected.
What people say isn’t what they do
The problem is, what you will hear from most focus groups is potential users’ projections of what they might do with your product in an ideal scenario where certain factors such as cost, maintenance, or other real-life demands don’t intrude. These projections will be amplified by the more vocal participants so that dissent is unlikely (see Solomon Asch’s work on conformity within groups – it’s been replicated in many cultures and situations).
Focus groups were designed for a different purpose
The end result that a focus group typically aims for is a go/no-go decision about a product. Even if the result could be trusted, it isn’t very useful to people who are trying to improve a product. User testing typically aims for a list of issues that should be fixed before a product is released. While some of these issues may be touched on during a focus group, the session is not designed to elicit these issues, nor to record them and assign any measure of severity to them.
Another typical use of focus groups is to gather information about a target audience. Here again the homogenizing power of the group can prevent nuances from surfacing. Much better to observe people doing the task you care about than to have them talk about it. If an interview is all you can manage, it’s typically much more useful to run one on one sessions, or small groups of co-workers, rather than bringing people together in a focus group environment.
Better uses for groups of users
So far I’ve been pretty negative about focus groups, especially as a usability or user research method. However, there are some very good reasons for getting people in a room together. Collaborative co-discovery, group card sorts, and testing of collaborative software or multi-player games all require a (small) group of users.
The difference between these methods and focus groups is that each of these methods either observes real behavior or asks behavioral questions – very similar to the preferred job interview technique – aimed at uncovering actual user interaction with the product or task you care about.
Bottom line: observe behavior, don’t encourage speculation
If you are performing group research, make sure you are getting truthful answers by observing or having users report on their real behavior in the situations you care most about. As soon as you ask someone to say what they might do in a hypothetical situation, you are getting out of the realms of data and into speculation.