Coding Horror: I Repeat: Do Not Listen to Your Users

Joel Liu's personal annotations on this page

Joel bookmarked on 2008-02-27 design product usability

I wrote the first version of Gmail in one day. It was not very impressive. All I did was stuff my own email into the Google Groups (Usenet) indexing engine. I sent it out to a few people for feedback, and they said that it was somewhat useful, but it would be better if it searched over their email instead of mine. That was version two. After I released that people started wanting the ability to respond to email as well. That was version three. That process went on for a couple of years inside of Google before we released to the world.

Startups don't have hundreds of internal users, so it's important to release to the world much sooner. When FriendFeed was semi-released (private beta) in October, the product was only about two months old (and 99.9% written by two people, Bret and Jim). We've made a lot of improvements since then, and the product that we have today is much better than what we would have built had we not launched. The reason? We have users, and we listen to them, and we see which things work and which don't.
Listening to users is a tricky thing. Users often don't know what they want, and even if they did, the communication is likely to get garbled somewhere between them and you. By no means should you ignore your users, though. Most people will silently and forever walk away if your software or website doesn't meet their needs. The users who care enough to give you feedback deserve your attention and respect. They're essentially taking it upon themselves to design your product. If you don't listen attentively and politely respond to all customer feedback, you're setting yourself up for eventual failure.
I think Paul had it right, but it's easy to miss. The relevant phrase in Paul's post is we see which things work, which implies measurement and correlation. There's no need to directly watch users (although it never hurts) when you have detailed logs showing what they actually did. Collect user feedback, then correlate it with data on what those users are actually doing:

Don't just implement feature requests from "user representatives" or "business analysts." The most common way to get usability wrong is to listen to what users say rather than actually watching what they do. Requirement specifications are always wrong. You must prototype the requirements quickly and show users something concrete to find out what they really need.

Acting on user feedback alone is questionable. No matter how well intentioned, you're guessing. Why guess when you can take actions based on cold, hard data? Acting on user feedback and detailed usage metrics for your application or website-- that's the gold standard.
We've traditionally relied on things like written feedback from players to help decide which improvements to focus on. More recently, Steam has allowed us to collect more information than was previously possible. TF2 includes a reporting mechanism which tells us details about how people are playing the game. We're sharing the data we collect because we think people will find it interesting, and because we expect to spot emergent problems earlier, and ultimately build better products and experiences as a result.
Make sure your application or website is capturing user activity in a useful, meaningful way. User feedback is important. Don't get me wrong. But never take action solely based on user feedback. Always have some kind of user activity data to corroborate and support the valuable user feedback you're getting. Ignoring your user feedback may be setting yourself up for eventual failure, but blindly acting on every user request is certain failure.

This link has been bookmarked by 20 people . It was first bookmarked on 26 Feb 2008, by lilin_fr.

30 Dec 08

Chris Smith
imported
29 Dec 08

Jeremy Gollehon
SoftwareDesign Usability
- It's rude not to listen to your users. So how do we reconcile this with the first rule of usability-- Don't Listen to Users?
  
  To discover which designs work best, watch users as they attempt to perform tasks with the user interface. This method is so simple that many people overlook it, assuming that there must be something more to usability testing. [It] boils down to the basic rules of usability:
  
  Watch what people actually do.
  Do not believe what people say they do.
  Definitely don't believe what people predict they may do in the future.
09 Apr 08

Jan Gondol
- Listening to users is a tricky thing. Users often don't know what they want, and even if they did, the communication is likely to get garbled somewhere between them and you. By no means should you ignore your users, though. Most people will silently and forever walk away if your software or website doesn't meet their needs. The users who care enough to give you feedback deserve your attention and respect. They're essentially taking it upon themselves to design your product. If you don't listen attentively and politely respond to all customer feedback, you're setting yourself up for eventual failure.
- basic rules of usability:
  
  Watch what people actually do.
  Do not believe what people say they do.
  Definitely don't believe what people predict they may do in the future.
- 3 more annotations...
- - no need to directly watch users (although it never hurts) when you have detailed logs showing what they actually did
  - Make sure your application or website is capturing user activity in a useful, meaningful way. User feedback is important. Don't get me wrong. But never take action solely based on user feedback.
  - Ignoring your user feedback may be setting yourself up for eventual failure, but blindly acting on every user request is certain failure.
20 Mar 08

Ryan Nance
design development for:slipperypig for:teian123 programming software testing ui usability users
04 Mar 08

Susan Smith
usability web design mistakes
Lois Lacroux
usability
- Listening to users is a tricky thing. Users often don't know what they want, and even if they did, the communication is likely to get garbled somewhere between them and you. By no means should you ignore your users, though. Most people will silently and forever walk away if your software or website doesn't meet their needs.
- To discover which designs work best, watch users as they attempt to perform tasks with the user interface. This method is so simple that many people overlook it, assuming that there must be something more to usability testing. [It] boils down to the basic rules of usability:
  
  Watch what people actually do.
  Do not believe what people say they do.
  Definitely don't believe what people predict they may do in the future.
03 Mar 08

Perry Branch
data design personas research visualization
- Acting on user feedback and detailed usage metrics for your application or website-- that's the gold standard
- Make sure your application or website is capturing user activity in a useful, meaningful way.
- 1 more annotations...
- - Always have some kind of user activity data to corroborate and support the valuable user feedback you're getting.
02 Mar 08

Julio B
28 Feb 08

Steve Lynch
feedback is still the key - systems theory

usability openness design feedback forblog
27 Feb 08

JingleMan Sweep
agile architecture business collection data users feedback gmail google development interesting management process product programming software website usability delicious
Colas Nahaboo
usability
Arrix Z
I Repeat: Do Not Listen to Your Users
Brent Sordyl
Listening to users is a tricky thing. Users often don't know what they want, and even if they did, the communication is likely to get garbled somewhere between them and you. By no means should you ignore your users, though.

design usability testing rubyonrails
Joel Liu
design product usability
- I wrote the first version of Gmail in one day. It was not very impressive. All I did was stuff my own email into the Google Groups (Usenet) indexing engine. I sent it out to a few people for feedback, and they said that it was somewhat useful, but it would be better if it searched over their email instead of mine. That was version two. After I released that people started wanting the ability to respond to email as well. That was version three. That process went on for a couple of years inside of Google before we released to the world.
  
  Startups don't have hundreds of internal users, so it's important to release to the world much sooner. When FriendFeed was semi-released (private beta) in October, the product was only about two months old (and 99.9% written by two people, Bret and Jim). We've made a lot of improvements since then, and the product that we have today is much better than what we would have built had we not launched. The reason? We have users, and we listen to them, and we see which things work and which don't.
- Listening to users is a tricky thing. Users often don't know what they want, and even if they did, the communication is likely to get garbled somewhere between them and you. By no means should you ignore your users, though. Most people will silently and forever walk away if your software or website doesn't meet their needs. The users who care enough to give you feedback deserve your attention and respect. They're essentially taking it upon themselves to design your product. If you don't listen attentively and politely respond to all customer feedback, you're setting yourself up for eventual failure.
- 3 more annotations...
- - I think Paul had it right, but it's easy to miss. The relevant phrase in Paul's post is we see which things work, which implies measurement and correlation. There's no need to directly watch users (although it never hurts) when you have detailed logs showing what they actually did. Collect user feedback, then correlate it with data on what those users are actually doing:
    
    Don't just implement feature requests from "user representatives" or "business analysts." The most common way to get usability wrong is to listen to what users say rather than actually watching what they do. Requirement specifications are always wrong. You must prototype the requirements quickly and show users something concrete to find out what they really need.
    
    Acting on user feedback alone is questionable. No matter how well intentioned, you're guessing. Why guess when you can take actions based on cold, hard data? Acting on user feedback and detailed usage metrics for your application or website-- that's the gold standard.
  - We've traditionally relied on things like written feedback from players to help decide which improvements to focus on. More recently, Steam has allowed us to collect more information than was previously possible. TF2 includes a reporting mechanism which tells us details about how people are playing the game. We're sharing the data we collect because we think people will find it interesting, and because we expect to spot emergent problems earlier, and ultimately build better products and experiences as a result.
  - Make sure your application or website is capturing user activity in a useful, meaningful way. User feedback is important. Don't get me wrong. But never take action solely based on user feedback. Always have some kind of user activity data to corroborate and support the valuable user feedback you're getting. Ignoring your user feedback may be setting yourself up for eventual failure, but blindly acting on every user request is certain failure.
26 Feb 08

lilin_fr
- I Repeat: Do Not Listen to Your Users
- But what resolutions do most people actually play at?
Ryan Baker-Smith
Web.Design.General
alan le
testing usability