Sometimes you can’t do all the usability work yourself. Either there’s too much work, it’s highly boring work, it’s highly specialist work, or the whole project is being outsourced. Here are some tips for choosing and working with vendors.
Yes, I’m a usability vendor. I’ve also previously been in the position of having to manage usability vendors both for on-site agency style work and for contracts with specific deliverables. This experience has shown me that a little bit of up-front communication can make things much easier for both the hiring party and the vendor.
When do you need vendors?
There are three main situations when you’ll be looking to vendors to help you out.
- Volume of work: you have too much on your plate and want either short-term or long-term assistance with running studies or other usability functions.
- Nature of work: either you have a bunch of dull and repetitive work that just needs to get done (benchmarking studies, for instance), or you have a need for highly specialist skills (eyetracking work) or you are buying in training or coaching so that your team can improve. You may also want to use vendors when it is important that the work is conducted impartially – for instance when comparing your product with a competitor’s.
- Team mandates it: if the product team is outsourcing work, hopefully they’ve also specified some level of usability or user experience requirements.
What type of vendor do you need?
What stage in the dev process is the team at?
- Concept: Use an experienced vendor who can use advanced methods to provide good interpretation of user needs and help the team form a strategy.
- Early development: Use an experienced vendor for concept testing (paper prototypes, site visits, etc.) and a cheaper one for lab-based studies using early code.
- Close to finished (beta): Use a cheaper vendor who can churn through participants for verification and can find show-stopper issues.
- Post-release: Use a cheaper vendor for benchmarking, a more experienced one for early work on the next product cycle.
What type of application/service are you building?
- Standard product, no particularly new concepts or interaction styles: Use a cheaper vendor who is confident with traditional usability testing techniques.
- Unusual, new, or different interaction styles: Work with a more experienced vendor to create suitable usability testing strategies for the novel areas. Later you may be able to transfer this work to a cheaper vendor resource.
What type of management resource can you offer?
- We have on-site UX managers: Cheaper vendors tend to need more oversight. An agency usability tester will need daily management, typically from someone very familiar with the usability role.
- We need a self-starter who can manage *us*: More experienced vendors will be more self-sufficient. They will ask for the resources they need and provide direction to the team rather than needing to be told what to do.
What type of budget do you have?
- User research should account for around 10% of total budget. That includes early concept research and later lab testing.
- Counterintuitively the less money you have, the more likely you are to benefit from hiring a more experienced vendor. They will provide deeper and longer lasting insights rather than just telling you how many people successfully used your current prototype.
Or to summarize, use a more experienced vendor when you need insights, use a cheaper one when you just need verification.
Creating the correct RFQ/brief and contract
The biggest determinant of success with any piece of vendor work is clear goals. It’s your job to create a clear RFP/RFQ which states the work and what constitutes successful completion – including UX goals. If you’re not into big documents like requests for proposals, then at least say what it is that you want to get out of the engagement, phrased in terms of the user experience.
Having this document allows you to measure different vendors’ proposals against some set goals. You’ll soon see which vendors actually understand and care about user experience, because their responses will include a description of how they intend to measure that experience.
You can also carrying these requirements forward into a contract, making usability a core component of the success of the project. Good vendors will understand how to measure usability as a success criterion.
There’s a side benefit to this approach. It makes you think about what you need up front rather than being led by vendor who may have a different focus to you. The vendor can still be creative in how they meet that need, and you can use your conversations with vendors to help you refine what it is that you want, but in the end you’ll benefit from thinking for yourself about what user experience goals you have for the project.
Items that aid clarity
Of course, there is an element of give and take. You can’t just make a whole bunch of demands on a vendor and expect them to deliver a great user experience without any guidance from you. There are several things that you can provide that will help a vendor to deliver to your expectations.
- Documented UX standards – whatever standards you work to within your organization. If they aren’t documented, there’s no way you can expect a vendor to follow them.
- Style guide – what tone should text have? How and where should company logos be used? What are the agreed colors to be used in the application? Do labels go above or to the left of form fields? Without a style guide, it’s hard to enforce consistency standards.
- Pattern library or Design System – if you have a set of interface elements that you consistently re-use, like a common sign-in module, a standard way of collecting customer data, or even just a common design for your search box, then share these with your vendor.
- Performance requirements – what constitutes an acceptable interface load time? Reaction time? Data download time? What percentage of errors or abandonments is acceptable? What do you consider to be an acceptable level of user satisfaction?
When the work starts
When the work starts, ask for regular updates on the vendor’s progress against your agreed goals. If the vendor can’t measure this progress, stop the work. So long as the goals were agreed in the contract, it’s perfectly reasonable to expect interim status reports.
Vendors who are undertaking research for you should be analyzing the results they get as they go along. If you’re paying for a large independent competitive benchmark, you need to know early on whether there’s a sufficient difference between the products for it to be worthwhile completing the research work. The vendor’s study design should be targeted at giving you this early feedback rather than at maximizing their income.
Vendors that leave usability measurement until they are done with a piece of development work aren’t going to be in a position to make deep changes based on their findings. In contrast, vendors who measure consistently as they go along will be able to make frequent course changes and deliver a better overal solution based on consistent user feedback.
If you’ve specified usability goals with vendors it’s much easier to hold them to a user experience standard. In the case where vendors are building an entire system for you, it’s also worth specifying what the penalty is for failing to meet user experience goals. You can tie UX goals to payment in the same way as you would with Quality Assurance metrics.