The 4 components of product quality: performance, bugs, completeness, and consistency

In every team I’ve been on, we’ve always had long-running discussions (sometimes edging into an argument!) about whether our product is the right level of quality.  Emotions run high — on one side, no one likes being accused by their peers of accepting low quality;  on the other, no one likes signing their name to something that doesn’t meet their personal quality bar.

Of course, the problem here is that every one of us has a different idea of what “quality” means.  For some, it means fewer customer support tickets.  For others, it means more bugs being caught before making it to production.  Other folks think it means polished animations and sparkles of fun inside the experience.

I think of quality instead as creating a functional, usable experience, much like a utility in the physical world.  If I flip a switch, do the lights go on?  

And just like with a utility, I’ve found that quality is composed of 4 mostly measurable components.

1. Performance.  I lived in an apartment where the entry lights took 5 seconds to turn on, and constantly found myself impatient even for that short delay.  Speed is unexpectedly magical —  when things are fast they feel tailored and responsive, and when they’re slow they’re disproportionately frustrating. And speed often has business results — I’ve seen customers place more orders, read more stories, and send more messages when their experience is faster.  

2. Bugs.  How would you feel if your lights stopped working at random times — while you’re deep in a conversation with a friend, or in the middle of eating dinner?  Tracking bugs and setting up test infra makes it a lot easier to know where and how often we’re letting down customers who are just trying to get something done.

3. Completeness.  Have you ever walked into a new office or conference room and discovered the lamps have been removed or not yet installed, and you'll just have to manage without light for a while?  Supporting every part of what the customer is trying to do is a key part of delivering quality.  I try to do this by identifying the most important end-to-end flows and making sure that they’re complete, and reading customer support tickets and research to understand where there are gaps.

4. UX consistency.  Imagine if every room had a different way to turn on the lights.  Sometimes there’s a switch on the wall by the door, sometimes it’s in the middle of the floor, sometimes it’s attached to the ceiling fan.  It’s doable, but harder — especially when you’re new.  Consistency makes it easier for every customer to succeed.  This could mean investing in design systems and common components, or just clear design principles for how we will build.

Of course, focusing on quality alone doesn’t make your product successful. I still have to build the functionality and feeling I’m aiming for. But this creates a solid foundation to build on top of.

These 4 components of quality have proven to be straightforward, independent, mostly measurable streams to track.  There’s no need for a “grand quality metric” which combines them, and no complex coordination system required to manage all these in lockstep.  Customers are happy when any of these improve, so I can just make progress on each of them individually whenever it makes sense.

Every time I’ve focused on these, my product has intuitively gotten better.  And it gives the whole team a sense for how we can work together to improve quality, in a way that's deeply felt by customers.