Escaping the Build Trap, by Melissa Perri
Discover more from Evan’s Notes
Escaping the Build Trap, by Melissa Perri
My detailed notes on how to escape a common failure mode of product organizations: becoming a feature factory.
This is another one of the “must reads” for product managers seeking to do meaningful work. Some of my favorite parts:
Her juxtaposition of bad product manager archetypes (chapter 6) with a great product manager (chapter 7)
How she explains the relative balance between tactical, operational, and strategic tasks for different levels of product manager (chapter 8)
How she incorporates other valuable frameworks, in particular Stephen Bungay’s “effects gap” framework to explain how strategic plans usually fail (chapter 11)
How she riffs off the Toyota’s approach to continuous improvement for her product process framework (the product kata, see chapter 15)
Her fantastic questions to ask during an interview to understand whether a team is product-led or not (see Appendix).
There is a lot to unpack in this summary, and I hope you enjoy the detail. The book goes into a lot more, and includes valuable examples to elaborate on the notes. I highly recommend any product leader to have their own copy. Find it at Amazon or Bookshop.
Preface
A true product-led organization sustains value creation by: a) having product manager roles with the right responsibilities and structure; b) enabling product managers with a strategy that promotes good decision making; c) determining what products to build through experimentation & optimization; and d) supporting a thriving product culture with good organizational policies, culture, and rewards.
PART I: THE BUILD TRAP
"The build trap is when organizations become stuck measuring their success by outputs rather than outcomes." This happens for a variety of different reasons, and fixing it requires a holistic approach. [Note: the author weaves in a fictional example of Marquetly as they build a new educational platform and transform from a feature-led to product-led company.]
Chapter 1: The Value Exchange System
The build trap emerges when companies associate value with the number of things they produce (features shipped, etc., easier to measure) instead of the outcomes they want to create. The customer only realizes value if their problems are solved, which then leads to desired business outcomes (increased sales, etc.) (see Fig. 1-2). To do this, the company must invest in and organize around a deep understanding of their customers' problems. Doing so creates a Value Exchange System (Fig. 1-3). It’s only possible to measure outcomes when you first understand what customers truly want or need.
"Products and services are not inherently valuable. It’s what they do for the customer or user that has the value — solving a problem, for example, or fulfilling a desire or need."
Chapter 2: Constraints on the Value Exchange System
Customers are influenced by their community, technology they use, and options they have in the market. Customer needs constantly evolve as their circumstances and expectations change. Companies also have constraints (see Figure 1-3), though they also have control over whether they allocate time to experiment and/or iterate on their products. Remember: if you don't measure the impact that your releases have, you will never see the connection between your work and the ultimate outcomes you're seeking.
Chapter 3: Projects vs. Products vs. Services
Some definitions:
Products: something that can deliver value repeatedly to customers or users, without “requiring the company to build something new every time.”
Services: something that primarily requires human labor to deliver value. Note that services can be productized, but if a person is needed to execute each time, it's a service.
Project: "a discrete scope of work that has a particular aim. It usually has a deadline, milestones, and specific outputs that will be delivered.” These are an important part of product development, but thinking only in this way is potentially damaging.
Chapter 4: The Product-Led Organization
The gold standard is the product-led organization, which scales and grows through software products, optimizing them until they reach desired outcomes. Some contrasting examples:
Sales-led have product strategies defined by contract negotiations. Many small companies start off as sales-led, which is okay, but there's a critical moment in growth when you either become a product company or a bespoke agency. This is hard if the sales organization is always running ahead of product.
Visionary-led (like Apple, with Steve Jobs) are rare and not sustainable. "Innovation needs to be baked into the system so that one person is not the weakest link."
Technology-led have product strategies driven by the latest and coolest technologies. These typically suffer from a lack of market-facing, value-led strategy. They risk spinning their wheels, producing cool things with few buyers.
Product-led companies optimize for their business outcomes, align their product strategy to these goals, and then prioritize the most effective projects that will help develop those products into sustainable drivers of growth.
Chapter 5: What We Know and What We Don't
Product development is filled with uncertainty. We need to separate facts we know from things we need to learn. Product management is the domain of recognizing and investigating the known unknowns (the "questions" portion) and of reducing the universe of unknown unknowns (the "discovery" portion).
PART II: THE ROLE OF THE PRODUCT MANAGER
Chapter 6: Bad Product Manager Archetypes
Most people who learn product "on the job" are learning within a waterfall environment: they collect inputs and requests from stakeholders, prioritize them as projects, organize them into requirements or specs, collect the designs, and then work through acceptance testing with engineers to make sure it's delivered bug-free. Agile is a software development process that largely ignores the top-of-funnel part of product management wherein ideas are generated and validated. Some bad archetypes of PMs:
The Mini-CEO is often an excuse for being an asshole. PMs have influence, not the ability to fire people or unilaterally change directions. You need to win over your team, since your job is to produce value, not develop your own ideas.
The Waiter is the PM who is, at heart, a people-pleaser and an order taker. There is no goal, no vision, no decision making involved. They lead to the "product death cycle" (Figure 6-1) of delivering oodles of features which have limited to no business impact. Some of this reactive mindset is learned helplessness. Pushing back is essential to building a successful product.
The Former Project Manager is the PM who is often more concerned with the "When" instead of the "Why" and is often paired with the Waiter role. They get stuck in the death cycle too.
Chapter 7: A Great Product Manager
The real role of the product manager in the organization is to (work with / influence) a team to create the right product that balances meeting business needs with solving user problems. At the most basic level, a product manager reduces risk by focusing on learning. More specifically:
They own the "why" of what they are building, work with their teams to develop the ideas, and then jump into the requirements to make sure that the product being created achieves the goals of the customer, user, and business.
They connect the dots between customer research, expert information, market research, business direction, experimental results, and data analysis to frame the questions and conduct discovery around unknown unknowns.
They harness the collective knowledge of the business, technology, and design departments. Product management itself is a domain wherein you can develop expertise ... and hiring for specific market or technology background can be detrimental at times. They must be tech- design- and market-literate to be able to discuss and weigh tradeoffs. They do not need to be fluent.
They ask "why," especially around each of the dimensions that could kill the project or make it worthless in the end. "Why even do this project?" or "What does success look like?"
They hold time for product vision and research work to figure out how to marry the business goals with customer goals to achieve value.
Perri summarizes it best: "With a good strategy framework in place and ruthless prioritization around a few key goals, one person can effectively talk to customers, understand their problems, and help to define the solutions with the team. The amount of external versus internal work will shift, depending on the maturity and success of your product. But, you should never be doing all this work at once."
Chapter 8: The Product Management Career Path
Different levels of PMs have a changing balance of a) tactical work (shorter-term actions to deliver features, scope work with developers and designers, and crunching data); b) strategic work (positioning the product and company to win in the market and achieve goals, figuring out how to get to the future state of the product); and c) operational work (tying strategy back to tactical work through roadmaps and by generating alignment).
No matter what level you are, the foundational skills are still working with a development team, understanding user needs and problems, measuring data, understanding the technical implications, knowing how user experience can impact user value, and connecting that back to the business goals.
Associate PM: [PSA: the industry needs more entry level product roles for new people to learn the discipline.]
Product Manager: Working within a quarterly focus, with responsibility for a feature or a set of features. Be careful of focusing too much on operational matters, so push as much project management as possible to the team and carve out time to be strategic.
Senior PM: Similar as PM, but with added scope or complexity. Still focused on building products instead of growing a team. Similar to the architect role in development, where someone will focus on "laying the development structure and scaling it." Senior PMs are critical to success of companies due to their ability to operate independently.
Director of Product: Working within the time horizon of a year, focused on promoting strategic alignment and operational efficiency, and connecting their product group back to the broader product / portfolio vision.
VP of Product: Oversees strategy & operations for an entire product line, and often responsible for the financial success of their product line, not just the delivery of features. Often the highest level in smaller companies with only one product. Needs to focus on the strategy and growing their team.
Chief Product Officer: Oversees the entire product portfolio to ensure they're working together to achieve company goals. Key traits: inspiring confidence, empathizing (always aligning goals), and being relentless and resilient (digging in to find what's working and what's not).
Chapter 9: Organizing Your Teams
A common mistake is to organize around technical components or features. It's often better to organize around value streams, which are "all of the activities needed to deliver value to the customer." More specifically, work backward from your customer or user and the value you are providing to them. Aim to organize your teams so you can provide more value to users, faster than today. Likewise, it’s useful to organize teams around your strategy so that they have “room to manage toward an entire outcome-oriented goal.”
PART III: STRATEGY
[Story of Netflix ruthlessly following their strategy, even spinning out Roku because it wasn't fully aligned.]
Chapter 10: What is Strategy?
Your strategy is not your plan. "Strategy is a deployable decision-making framework, enabling action to achieve desired outcomes, constrained by current capabilities, coherently aligned to the existing context." (From Stephen Bungay's The Art of Action).
Chapter 11: Strategic Gaps
Approaching strategy as a plan (and leading through authority) often fails to achieve what we expect. This is because of three major gaps that cause friction within the organization (figure 11-3). The antidote to all of this (and the most important way to scale an organization) is to "enable action through autonomous teams."
We try to fill the knowledge gap by demanding more detailed information. Instead, management should limit itself to defining and communicating the strategic intent (a.k.a. the goals) of the business. It's a good thing when PMs describe uncertainty (e.g. "We don't know what the problem is yet, but we're going to discover it and solve it"). Avoid demanding more detailed information.
We try to fill the alignment gap by providing more and more detailed instructions. Instead, management "should allow each level within the company to define how it will achieve the intent of the next level up." Management should avoid dictating or committing features on the roadmap, and instead articulate why and allow the teams to figure out how and report back.
We try to fill the effects gap by putting more controls in place, which is the worst thing you can do. Instead, management needs to "give individual and teams the freedom to adjust their actions so that they are in line with their goals."
Chapter 12: Creating a Good Strategic Framework
A good strategic framework is one that a) enables your team to be sufficiently constrained to be able to make good decisions (e.g. constraints force alignment with the strategy); b) allows learnings to flow up and down so that all levels of the strategy can adjust as needed; c) acknowledges your current strategy as a set of "bets" you're making about what will help you succeed. Think carefully about all factors: how you create your strategy, how you deploy your strategy, and how you continuously improve your strategy. Other Details:
Distinguish between your company's operational framework (keeping the day-to-day stuff moving) from the strategic framework (how you realize the vision by bringing products and services to market).
Strategy creation: "The process of figuring out which direction the company should act upon and of developing the framework in which people make decisions." This requires first understanding the vision, and then mapping the obstacles in the way.
Strategy deployment: "Strategies are interconnecting stories told throughout the organization that explain the objective and outcomes, tailored to a specific time frame." Strategy deployment is how you communicate and align these narratives.
Example frameworks for strategy deployment, all designed to "set the direction for each level of an organization so they can act."
DIBBs (Spotify) --> Data, Insights, Beliefs, and Bets. The first three inform the piece of work framed as a "Bet" to ensure it sets appropriate expectations.
OKRs (Google, others) --> Objectives & Key Results.
BPM (Twilio, via First Round) --> Big Picture, Priorities, Measures. (Also, make sure you argue more during planning).
Red flags: When the leadership team goes into "planning mode" and emerges with a list of features to build, which then get handed to the product managers. Or when OKRs are written in output-oriented language, e.g.: "Ship the first version of this product" (Objective) with "Deliver by June 2021" (Key Result).
[The author draws inspiration from Toyota's "Continuous Improvement" approach for her Product Kata, explained soon.]
Chapter 13: Company-Level Vision and Strategic Intents
The company vision sets the ongoing direction and provides meaning for everything that follows. Strategic intents, anchored to this particular moment, are the handful of "concise, outcome-oriented goals which focus the company around how to reach the vision." Each strategic intent then becomes the north star for the related product initiatives which emerge from discovery & research to accomplish the goals. Other details:
Best if the mission (why the company exists) and the vision (where the company is going based on that purpose) are combined into a single statement with no fluffy language. It needs to take a stand and NOT be too broad.
Strategic intents are the "big leaps forward" with a timescale of 1-2 years. Keep the list small to maintain focus. (Getting the right level and number of these for your company is really important.) These need to be articulated not as individual features, but rather in the context of business value (e.g. "double revenue growth from individual users"). One intent may work for a small company, and three should be plenty for a large organization.
MECE framework for framing business value creation: increasing revenue, protecting revenue, reducing costs, or avoiding costs.
Netflix example vision statement: Becoming the best global entertainment distribution service, licensing entertainment content around the world, creating markets that are accessible to film makers, and helping content creators around the world to find a global audience.
Chapter 14: Product Vision and Portfolio
Product initiatives articulate the problems we will solve with our product to reach the business goal articulated as the strategic intent. Options are the bets - potential solutions - aligned to each product initiative. Product managers are responsible for keeping the initiatives & options aligned with the vision for an existing product or portfolio. Other details:
Being too prescriptive with the product vision stifles the innovation process. Solutions should “emerge from experimentation around solving problems for customers."
Example process: Amazon creates press release documents for every product vision. Each is a 1-2 page document describing the problem the user is facing and how the solution enables the user to solve that problem.
Netflix example: Strategic intent = lead the streaming market. Product initiative = enable people to watch Netflix on any device, wherever viewers wanted to. Options = Roku, Xbox app, partnerships with internet-enabled TVs, iPad App, etc.
VP of Product may not be the first person to set the product vision, but they are responsible for keeping everyone is aligned to the holistic vision (within the context of a single value chain).
Main questions for a CPO: How do all of our products work as a system to provide value to our customers? What unique value does each of the product lines offer that makes this a compelling system? What overall values and guidelines should we consider when deciding on new product solutions? What should we stop doing or building because it does not serve this vision?
Lack of time to innovate is usually due to poor capacity planning and strategy creation. We make time to innovate by saying no to less important things.
PART IV: PRODUCT MANAGEMENT PROCESS
Chapter 15: The Product Kata
The habits of the Product Kata are a repeatable, systematic way to build products from a problem-solving standpoint. Do just enough to get to the goal you've set (this will prevent spreading yourself too thin). And focus your efforts on things that will make or break your core value proposition. (E.g. others have already solved basic checkout flows, so copy existing solutions, see whether it works for you, and iterate).
Some definitions for Figure 15-1:
Within steps 1-3, the management team engages within the level of Company Vision & Strategic Intent -> Current State of intents -> Product initiatives.
Within steps 1-3, the product teams engage within the level of Product initiative -> Current state of initiative/product -> Option goal
Within step 3, Option Goal = "next level of goals from the team."
Chapter 16: Understanding the Direction and Setting Success Metrics
Frame your product initiative in terms of the product metrics which tell you how healthy your product is within the context of the strategic intent. At any given time, choose whichever metrics are most directly connected to your strategic intent and have the biggest room for improved performance. Two popular product metrics frameworks are:
Pirate Metrics: Acquisition = users find you and sign-up. Activation = when they have a first post-sign-up experience of your product (e.g. adding their first 4 friends). Retention = means and rates of users returned. Referral = users tell others about you. Revenue = profit you gain. Your specific product may have these in a different order, e.g. revenue for a B2B product typically comes before users activate.
HEART Metrics: happiness (how satisfied they are with the product), engagement (how often users interact with the product), adoption (similar to activation above), retention (same as above), and task success (how easy it is for them to accomplish what is intended by the product).
Other best practices: a) avoid vanity metrics (which always go up); b) set-up metrics to be mutually destructive (so that a single targeted metric can’t be gamed in a way that hurts the business); and c) identify real-time proxies for lagging indicators (eg. retention is a lagging indicator, whereas activation, happiness, or engagement can give real-time information).
Chapter 17: Problem Exploration
The next step is to do generative research with real people to understand the user problems at the core of your product initiative. It takes discipline to fall in love with identifying and understanding the correct problems (and not jumping to the solutions). Other details:
Avoid evaluative research (e.g. usability testing of a prototype) during the problem identification stage. User testing is most helpful when optimizing an experience.
Red flag: Framing the problem as the lack of a particular feature.
Many seemingly straightforward features (like custom dashboards) have so many different potential approaches that the ONLY way to zero-in on the most compelling solution is to find the most compelling user problem(s) at the center.
There is always some way to talk with customers. Recruit sales people to be research spies or challenge bureaucracy if you need to.
Get creative to understand the real context of customers' interaction with your product. [Example of offering a concierge-style service to identify pain points of course creation.]
Chapter 18: Solution Exploration
The next step is to explore solutions, and it is important to build to learn (we are not yet building to earn). Create experiments to narrow down whether or not solutions are moving in the right direction. Make these quick, cheap, and tailored to whatever you need to learn next. Common types of exploratory experiments:
Concierge Test - where you deliver the end result manually (and customers know it's all manual). Working closely with the customer creates tight learning loops and helps you understand what customers really care about. Make sure to allocate enough time to deliver on what they need.
Wizard of Oz Test - where you deliver the end result manually but it looks like a finished product to the customer. Great for getting feedback at a little more scale than Concierge, and without the upfront "handshake" between you and customers (which may change customer behavior). Take care not to leave this up for too long ... don't treat it as a product launch! This can be paired with an A/B test if you have enough traffic. For example, Zappos was tested as a WordPress site with photos of shoes, which the founder would then buy and mail to customers.
Concept Test - when you present lightweight mockups which illustrate solution concepts to customers to get feedback, either through user interviews or demo videos. These can be made more evaluative when paired with strong pass-or-fail criteria, e.g. asking for upfront payment or some other commitment. Example: Dropbox's faked demo video showing how the file sharing service would work.
Other details:
Danger zone: diving too deep into solving problems (like video editing for the example Marquetly team) which are not a core value proposition to the company. Instead, invest enough effort to understand the user problems so you can solve it in a scalable way that makes sense for your company (e.g. buying the functionality from elsewhere), and then get back to focusing on your core.
Danger zone: defining the "MVP" as the first release, especially if that leads to cutting corners to get the feature out faster. Prioritize learning over releasing to maintain focus on outcomes.
Experiments focused on learning are not intended to be long-lasting solutions. Make sure to know exactly how you'll "close the loop" and end the experiment, and communicate these expectations to customers and internal teams.
When the underlying problem & solution is clearly understood (there is high certainty), there is no need for robust experimentation.
Solution experimentation is absolutely possible (and just as important) with complex products or internal tools. Remember: "If people can't figure out how to use the tools, it's on you (as the PM), not them."
Maintain notes on your "product kata" for each iteration... e.g. in a spreadsheet with the following columns: Current State, What to Learn, Next Step, Expected Result, Learned.
Chapter 19: Building and Optimizing Your Solution
Once you identify a promising direction for your solution, remember that the work is not finished until you have actually hit your goals. Some ways to keep focused:
A North Star Document provides context to a wide audience. It includes a) problem it is solving; b) proposed solution; c) solution factors which matter for success; d) outcomes the product will result in.
Story mapping breaks down the work by each desired action from the user standpoint. This encourages alignment and understanding as a scrum team delivering against the goals.
Cost of delay is an approach to prioritize features based on a combination of urgency and customer value. It's about reducing scope enough so you can capture the maximum value in the right time. E.g. every week faster we ship means another 100 invoices completed. For more details, see The Principles of Product Development Flow by Don Reinertsen or this article.
Definition of done should be based on actually achieving the outcomes you're targeting. It's not v1 of a release, but rather when you've iterated toward a solution that accomplishes what you've set out to accomplish.
“If it is high urgency, that means that every moment you do not ship that feature to customers, you are losing out on an opportunity to hit your goal — for example, if you are actively losing customers or revenue each week because you are not fulfilling a need. High value is about solving the strongest problems or desires for the customer."
PART V - THE PRODUCT-LED ORGANIZATION
Chapter 20: Outcome-Focused Communication
Many companies fail to make the transition to product-led due to lack of leadership buy-in to be more outcome-oriented. Find ways to communicate and talk about actual progress (e.g. based on outcomes) at different levels. Develop roadmaps as an "explanation of strategy and the current stage of your product." Other details:
Red flag: When a company gets too much satisfaction from shipping features and interpreting that as progress. It takes discipline to stay focused on outcomes.
Progress must be shared broadly to keep the knowledge gap as small as possible. Suggested levels:
Quarterly business reviews --> aligns with the strategic intents. This is NOT a meeting to discuss details (or prioritize) product initiatives. This is for the senior leadership team to discuss key financial metrics (revenue, churn, etc.) and "how the outcomes of product initiatives have furthered strategic intents."
Product initiative reviews --> aligns with product initiatives, also quarterly, though staggered in off months with business reviews. This is for the product side to review "progress of options against product initiatives and adjust strategy accordingly."
Release reviews --> aligns with options. Teams get to show off the hard work done and talk about success metrics. This is monthly, before features go out and become generally available.
Roadmaps must be living documents (not dead Gantt charts) with the following information: Theme, Hypothesis, Goals & Success Metrics, Stage of Development, and Important Milestones. (For more information on roadmapping, see Product Roadmaps Relaunched, by Bruce McCarthy.)
Consider standardizing your "Stages of Development" so that it's easily understood by everyone. E.g. Experiment, Alpha, Beta, and Generally Available (GA).
The author strongly recommends creating a product operations team, reporting to the CPO, which handles "streamlining all operational and process work that product teams need to be successful." This lean team would focus on a) streamlining data collection to track progress on outcomes; b) report on goals, outcomes, etc. across the product organization; c) standardize cross-team product processes; and more. (See book for details.)
Chapter 21: Rewards and Incentives
Be very careful to ensure your reward structures incentivize the right behavior. Red flag if any PM is ever stuck saying "It doesn't matter what the goal is. We just have to deliver this feature." If you find yourself as one of those PMs, push back! This includes when you find yourself working on a product that should be sunset because it no longer aligns with company strategy, even though there's a whole team aligned with it today.
Chapter 22: Safety and Learning
Some companies fail to try new things because there's "not enough safety in the organization to fail and learn." The key is to fail within the experiment stage so we have plenty of time to adjust our approach and try different options to drive the product initiative forward. Other details:
It can be helpful to set boundaries for experimentation, as in "you can only spend $100K on this experiment (all-in), after which let's review what we've learned and we'll think about investing more."
Explain the results of experimentation not just in opportunities unlocked, but also risks mitigated and money saved. E.g. example from the author where "we just saved $250K and increased sales 3x over this other way of working!" This framing leads to buy-in and trust.
Chapter 23: Budgeting
Some companies fail to break out of the output-over-outcome mindset because their approach to budgeting is too rigid or divorced from an experimentation-based product development process. Red flags if teams get funded based on pie-in-the-sky business cases based on little data and are then fixed until the next round of planning, or if teams are penalized if they don't "spend all their money." Instead, fund product development like a VC, where more and more money is allocated as opportunities become more validated.
Chapter 24: Customer Centricity
Becoming truly product-led means being customer centric. Put yourself in the customer's shoes and ask "What would make my customers happy and move our business forward?" Within the value exchange, you really need to understand the value on the customer side. Learn from companies which take more extreme/opinionated stances in this way, e.g. Amazon or John Deere. Always allocate time to visit or meet your customers.
Chapter 25: Marquetly: the Product-Led Company
[The Author wraps-up her example used within the book, and then shares some closing advice she learned about being an effective PM]:
[Starting in her career] "I learned that my role was not that of the big idea generator but that of the bad idea terminator. I needed to learn to be humble and to gain the support and buy-in of my team in order to make great products. Experimenting with my team taught me the power of data. Data beats any opinion every time."
[Becoming more senior] "I learned that having a good strategic framework could make or break a company. That if you did not judge people for success by outcomes, you would never achieve those outcomes."
[Being a consultant] "I learned about the power of personalities in an organization. People will get in the way of a good product every time. Even if it is the best idea for the company, if it doesn’t meet the personal agendas of senior stakeholders, it can be squashed. To mitigate that risk, you need to deeply understand what motivates people and to know how you can address their personal motivations by introducing information and data that wins them over."
Appendix A: Six Questions to Determine Whether a Company Is Product-Led
Who came up with the last feature or product idea you built? —> Ideally you see confusion. Red flags if they can't take ownership for it, or if they don't know why they're doing it. Or if the idea came from the CEO w/o explanation.
What was the last product you decided to kill? —> Another red flag is a company that can't kill any bad ideas, whether due to immovable budgeting, overcommitments to customers, or no pushback to management.
When's the last time you talked with your customers? —> Red flag if they can't talk with the customers or don't have any process in place for it.
What is your goal? —> Red flag if they struggle to articulate their goal (speaks to poor product management at the organizational level), or if it's output oriented and framed around dates instead of impact.
What are you currently working on? —> Good sign if they're excited, can articulate the big-picture goals, etc.
What are your product managers like? —> Good sign if development and UX people say, “I love my product manager. She has clear direction, communicates well, and helps us stay focused on the goals and problems.”
Escaping the Build Trap, by Melissa Perri
A good book indeed! Good summary.