Good Product Team/Bad Product Team

As VP of Product at Amplitude, I get the opportunity to work with hundreds of different products teams every year — ranging from startups with only a handful of engineers to large enterprises with thousands of PMs. Each of them works with Amplitude because they believe we will help them build a better product.

But this naturally raises a simple question: why do companies need help building better products?

There are many reasons for this, but at its core it comes down to the fact that building product is hard, and most companies are not great at it. And the traditional ways of building and delivering product just don’t cut it in today’s product-led era.

So in the mode of Ben Horowitz’s classic essay Good PM/Bad PM, I’ve captured my thoughts on what I believe makes a good product team vs a bad product team.

On solving customer’s problems…

Good product teams are customer obsessed. They interact with their customers directly to get first hand feedback. They bring the voice of the user right into the heart of decision-making at every stage of the process. They ensure that everyone in product development — PM, Design and Engineering — are experts on the customer.

Bad product teams outsource customer understanding. They rely on second-hand feedback and see “The Business” as a proxy for customers, gathering requirements through antiquated methods like MRDs. They rely upon analysts to answer every question regarding user behavior, which delays decisions by days or even weeks of time. PMs will then ask for more dashboards, but whenever they need to explain why a metric changed they say they’ll “get back to that next meeting” (and they never do).

Good product teams learn how to fish vs having analysts fish for them. Bad product teams make decisions based on a best guess from recent and anecdotal evidence, reserving analytics resources only for ‘mission critical’ questions. They rely on the noisiest qualitative sources, like sales, customer support or app reviews, and can’t see the forest from the trees.

Good product teams don’t build for their customer today, they build for who their customer will become. They use data to inform product decisions, not validate decisions they’ve already made. They use product analytics to know how users are actually using their product, not just rely on their best guess assumptions. They go beyond vanity metrics and have product analytics solutions that allow them to dive deep into their data to understand their customer’s journey.

On testing and refining…

Bad product teams lack a product strategy, or if they have one that strategy is not clearly delivered throughout their organization. They have a culture of being very reactive to the whims of upper management, which destroys entrepreneurial spirit and drive. They require 6 levels of reviews on every decision before they move forward because of dependencies and ‘turf wars’.

Good product teams understand the value of focus. They understand that having a product strategy means saying no many more times than saying yes. They have intention in their actions based on their core understanding of the problems they need to solve for. They set an aggressive 10x vision, but give the teams closest to the customer autonomy to figure out how to achieve that vision.

Bad product teams use agile as an excuse to not have a vision.

Good product teams execute against their vision in 10% increments. They know that while you cannot substitute vision with data, you can better understand customer needs and test hypotheses about what customers want by utilizing data. They identify the riskiest assumptions and create experiments to test those assumptions. They build minimal valuable products that are narrative-complete, not feature complete. They constantly ask themselves if what they are focused on will have the biggest impact for their business.

Bad product teams claim because they’re agile they move quickly, but still take 3 months to release a feature. They work in sprints, but nothing reaches the customer at the end of the sprint. They say they are lean, but instead of building a skateboard, they build a car without a steering wheel.

Good product teams are optimized for learning: they understand that the first attempt is always wrong, and can iterate quickly because they have access to product analytics data to discover what is and isn’t working for the customer. They recognize that to innovate means taking on the risk of breaking what you have today. Bad product teams live in fear of losing what they have today.

Bad product teams make excuses to not ship quickly, and try to get it right on the first try. They say they celebrate failure, but don’t give people the room to explore. Good product teams experiment patiently, accept failures and double down when they see customer delight.

On measuring success and failure…

Good product teams measure themselves by outcomes; engagement, stickiness and retention, and can ultimately tie those to revenue generation. Bad product teams measure themselves by outputs; story point velocity, lines of code shipped or features delivered. Cost, schedule and scope rule the day.

Bad product teams follow a strict release schedule, and are always behind. Their PMs see impact as having a thoroughly documented user story. When they finally complete their project, they move onto the next thing and never look back because “they’re done”.

Good product teams know that product is always evolving — learning is never done and they put processes in place to make sure they come back to what’s important, even if that means deprecating a feature or product. Bad product teams instead operate in feature factories, constantly chasing the next shiny object in hopes of finding a silver bullet to save their product.

Good product teams understand that products are built with lead bullets — that you can’t 80/20 your way to everything and that there are certain things that you have to be better at than everyone else.

Bad product teams are organized in functional silos and point fingers at each other when “behind.” They lack clarity around role definition — PMs and Designers believe they need to “keep engineers busy.” Good product teams are organized around a shared view of the customer and common definitions of success. They clearly define who is accountable for what between PMs, Design and Engineering.

Good product teams articulate their goals ahead of time and share the impact that their work has against those goals (good and bad) to the entire company. Bad product teams celebrate shipping only, but don’t take the time to articulate impact. They believe that looking at product analytics is the responsibility of the PM team only, not the broader team.

Good product teams take the time to define a core set of critical metrics that matter for their customer. Everyone understands how their work impacts the broader goal for the company and what is most important to achieve. Bad product teams define every product metric as a KPI, effectively making nothing important.

Do you have examples from product teams you’ve been a part of? I’d love to hear them. Please share in the comments below.

Learn more about how to get into product management and product management best practices.

Comments

Sharon Sciammas: Thanks for sharing your experience I could not agree more. The question is how to you tranform bad behavior to a good one? most ppl what to their best. bad team do know that they are bad. How do you take a bad team and help them grow (especially when they are not aware of being bad)

Shane Williamson: Nailed it 🙂