Continuous Discovery Habits, by Teresa Torres
My detailed summary of Torres's practical handbook for highly effective product discovery. Each chapter is a gem and has potential to transform the way you work.
Continuous Discovery Habits reads like an operating manual for effective product discovery, and is the best I’ve read so far on that topic. Torres addresses confusion around different kinds of outcomes, assumption testing versus idea testing, ways to conduct effective stakeholder management, and more. Every chapter leaves the reader with something they can immediately bring to their team to work together more effectively.
If you enjoy my summary below, I highly recommend buying and reading the full book either via Bookshop or Amazon.
PART 1: WHAT IS CONTINUOUS DISCOVERY?
Chapter One: The What and Why of Continuous Discovery
Continuous discovery is a structured and sustainable approach to make sure you build things customers want while also creating value for your business. Prerequisite mindsets to be successful:
Outcome oriented: Outputs (features shipped) are less important than the impact you have in the lives of your customers and for your business.
Customer-centric: Meeting customer needs is paramount.
Collaborative: Work is done best cross-functionally, leveraging each perspective’s expertise and knowledge.
Visual: Visualizing what we know and what we think unlocks our immense capacity for spatial reasoning.
Experimental: Uncover the truth by scientifically testing assumptions and hypotheses against evidence.
Continuous: Iterate and evolve your work; it is never static.
These habits are intended to be shared by the product trio: the standard pairing of product manager, designer, and software engineer who make up the core of a product team. Each element is critical: "a) at a minimum, weekly touchpoints with customers; b) by the team building the product; c) where they conduct small research activities; d) in pursuit of a desired outcome.”
Chapter Two: A Common Framework for Continuous Discovery
It can be dangerous to take shortcuts and assume we know what customers will value. When delivering an outcome, engage directly with customers to understand their context. Conduct your research to figure out the best path (of many) to your desired outcome, and make sure your research will inform the decisions you’re making this week and not being done for the sake of doing research.
Some framing tips: First, focus on the "opportunities to intervene in our customers' lives in a positive way," since we don't just solve problems, we also address desires. Try out multiple framings, exploring how each impacts the solution space. Visualize your thinking through an opportunity solution tree, which a) starts at the singular desired outcome; b) maps the opportunity space; c) and eventually leads to the solution space; d) each of which has relevant assumption tests to drive evaluation.
Main benefits of using Opportunity Solution Trees (OSTs) are that they:
Resolve tension between business needs and customer needs by focusing on opportunities that align with the business outcome.
Build (and encourage collaboration around) a shared understanding of the different ways to reach your desired outcome.
Encourage a mindset of continuous improvement by focusing on smaller opportunities which all flow up to the broader outcome desired.
Unlock better decisions by comparing & contrasting opportunities instead of getting lost in complexity or distracted by what might feel pressing in the moment. Focusing on smaller sub-opportunities also helps prevent analysis paralysis, and frequent research activities help with quick course corrections based on what you learn.
Unlock faster learning cycles, since the opportunity and solution spaces evolve together. Anything we learn helps refine our understanding of the overarching opportunity).
Build confidence in knowing what to do next, especially when the OST is too broad (narrow your focus) or too shallow (do more discovery interviews to learn about your customers).
Simplify stakeholder management by showing how you're thinking about the overall outcome (big picture) while also communicating details about the specific parts you've explored so far.
PART 2: THE CONTINUOUS DISCOVERY HABITS
Chapter Three: Focusing on Outcomes Over Outputs
When done well, `managing by outcomes` gives teams the "autonomy, responsibility, and ownership to chart their own path." However, adopting an outcome mindset is hard because: a) outcomes are hard to measure (most are lagging indicators); and b) it’s hard to know how to impact an outcome. This is made easier when we distinguish between:
Business outcomes - metrics that matter for the business (retention, sales), though are often lagging and have multiple cross-functional teams contributing meaningfully toward them.
Product outcomes - metrics that product teams specifically influence. Research is often required to identify which product outcomes most meaningfully influence business outcomes.
Traction metrics - usage of specific features. Having an experienced product team focus on traction risks limiting the types of decisions they can make. These metrics can be great for junior product trios or for teams managing mature products with a specific traction metric known to be critical to company success and deserving of an optimization team.
It’s important for product teams to negotiate with the product leader (CPO, VP of Product) around their outcome. The leader contributes the “big picture strategic intent,” and the product team helps to weigh tradeoffs in ambition (how much you expect to move the outcome) vs. risk (chance of failure) vs. resources and other projects competing for attention. Some guidelines:
For teams asked to deliver outputs which don't work toward outcomes: Challenge the outputs by asking why they matter. Dig for business context and work to level up to product outcomes.
For teams stuck in one-way outcome setting: Collect business context from the product leader and discuss the links you see between product & business outcomes.
For teams already engaged in two-way negotiations: give yourself a pat on the back and then apply this chapter to make sure you understand what you're tackling (regarding product vs. business outcomes, learning vs. performance goals, etc.)
Author’s reflections on setting SMART (specific, measurable, achievable, relevant, time-bound) goals: this can create focus and inspire effort, but make sure the team believes it is achievable. If needed, consider setting an initial learning goal (e.g. investigating the product behaviors associated with churn or retention) which will inform a SMART performance goal later.
Anti-patterns to avoid: a) pursuing too many outcomes at once (this limits you to incremental progress only); b) ping-ponging from one outcome to another (you lose the benefits of the initial learning curve); c) setting outcomes as individuals (it's better to set one common outcome for the product team); d) letting outputs creep in (it can be challenging to avoid this trap); e) or "focusing on one outcome to the detriment of all else."
Discovering Opportunities
Chapter Four: Visualizing What You Know
Start by mapping the customer experience as it exists today related to your desired outcome. Tips:
Find the right scope for the experience map using your desired outcome. Look beyond your own service to capture the broader context of the customer's life (which brings inspiration from competing platforms). Really broad questions (e.g. "How do customers entertain themselves?") might be useful if your outcome is to explore adjacent market opportunities.
Draw the experience map, even with simple boxes and arrows and stick figures. Use as few words as possible (language is vague and may lead to people thinking they are in agreement). Drawing is more concrete and specific.
Have each member of the product trio map the customer experience on their own, then compare the results. This avoids groupthink and captures the diverse perspectives of your team. Don't advocate for a "best" map, but use each to clarify your thinking.
Synthesize the different perspectives into one shared experience map by: a) turning each individual map into a collection of nodes (distinct moment in time, action, or event) and links (which connect nodes together); b) putting them all together into one comprehensive map; c) collapsing similar nodes together (careful not to lose key detail here); d) determining links between each node (not just the happy path); and e) adding context (what the customer is thinking, feeling, or doing).
Anti-patterns to avoid: a) getting bogged down in endless debate (draw your differences instead of debating them); b) using words instead of visuals; c) moving forward as if your map is true (test your assumptions, more on this later); d) forgetting to refine and evolve your map as your learn more.
Chapter Five: Continuous Interviewing
Make a habit of conducting weekly customer interviews in which you collect stories which highlight unmet needs. "Distinguish between what you are trying to learn (your research questions) from what you ask in the interview (your interview questions)." Tailor the scope of each interview to whatever you need to learn about at this moment in time, and remember the fundamental research question at the core of any interview: "What needs, pain points, and. designs matter most to this customer?"
Investigative interviewing, if we’re not careful, leads to a lot of poor information. When asked even factual questions, we are prone to cognitive biases; we speculate about what we did, when, and why; we give answers influenced by our sense of identity rather than our actual behavior; we craft coherent stories to explain our behavior that may have nothing to do with reality. (All of this is magnified when we are trying to be helpful.) Daniel Kahneman explains that "confidence is a feeling, which reflects the coherence of the information and the cognitive ease of processing it." Not necessarily the truth.
The most reliable way to mitigate these limitations is by focusing on specific stories of their experience to capture their actual behavior, not perceived behavior. Ways to excavate the full story:
Explicitly invite participants to share their full story (and ask for missing details)
Use temporal prompts, such as "What happened first?" followed by "What happened next?" If they seem to skip steps, prompt with "What happened before that?"
Ask about supporting characters or challenges: "Who was with you? What challenges did you encounter? Did anyone help you?"
Steer them away from generalized statements. When they say "I usually do X" counter with the follow-up question: "In this specific example, what did you do?"
Synthesize your discoveries as you go with a snapshot of each interview. Include: a photo (to remember the stories); a memorable quote (ideally with vivid language of their own); quick facts for context (maybe related to how you segment customers); opportunities (any needs, pain points, or desires expressed during the interview ... NOT solutions); and interesting insights (any intriguing behaviors not yet representing a known need, pain point, or desire). Turn a feature / solution request into the underlying need by asking, "if you had that feature, what would it do for you?" For even more powerful pattern recognition and insights, draw the experience map as told by the participant.
Interviewing customers every week helps to keep your discovery process agile, since you can disprove one opportunity and already begin gathering information on the next within a few days. Tips to make this sustainable: a) automate the recruiting process (build recruitment into your app); b) enlist customer-facing colleagues to recruit for you; or c) create a customer advisory board where each participant is interviewed once a month. Have all members of the product trio participate in each interview, since it keeps you learning together (and you each bring your own perspective to bear).
Anti-patterns to avoid: a) relying one person to recruit and interview participants; b) digging into who, what, why, how, and when questions (focus instead on one or two story-based interview questions for each research question); c) interviewing only when you think you need it (a weekly habit is easier to maintain and keeps you agile); d) sharing what you learned by sending pages of notes and/or a recording (it’s not feasible to expect your colleagues to do the work of synthesis); e) stopping the whole process to synthesize a set of interviews (this assumes what you're doing has a start or stop).
Chapter Six: Mapping the Opportunity Space
Mapping the opportunity space makes it easier (and faster) to identify new opportunities and the product strategies to unlock them. Our goal is not to address all of the dozens of potential problems customers have, just those which drive our desired outcome. Maintaining this focus ensures our products are viable over the long run.
First, take an inventory of the possibilities through "systematic and protracted inquiry" (quoting educational philosopher John Dewey) so we do not jump into the first need we identify. Then, this structure "gets done, undone, and redone" (quoting cognitive psychologist Barbara Tversky) as our opportunity space changes as we learn. Some specific tips, some repeating earlier notes:
This is most intuitive as a tree (not just a list) with parent-child and sibling relationships. This organizes everything into smaller, solvable opportunities which you can iterate through quickly.
Organizing opportunity the solution tree effectively requires critical thinking (and is hard work)
With siblings: make them independent so you can address one without addressing another.
With parent-child: make it so the higher-level opportunity is addressed by any of the children.
Identify distinct branches based on the stories customers tell you about their experience.
Frame each opportunity as a "customer need, pain point, or desire and not a solution"
Prune opportunities which are either a) specific to a single customer and not many; or b) do not drive the desired outcome.
Work through each branch, adding more structure and taking care to word them specifically. Give it just enough structure to be valuable; don't worry about making it perfect.
Anti-patterns to avoid: a) framing opportunities from your company's perspective (and not the customer's); b) vertical opportunities (where a parent has one child, which has one child ... reframe as the broader need or find more siblings); c) opportunities have multiple parents (get more specific and break them apart if needed); d) opportunities are not specific (e.g. they represent themes, design guidelines, or sentiment); e) opportunities are solutions in disguise ("Customers can fast forward through commercials" --> "Customers don't like commercials"); f) capturing feelings as opportunities ("Customers are overwhelmed" --> instead, identify the cause of the feeling).
Chapter Seven: Prioritizing Opportunities, Not Solutions
"You are never one feature away from success, and you never will be." Escape the build trap (where you're always falling behind on your stack-ranked list of projects) by focusing on one target opportunity at a time (limit your work in progress). Use the opportunity tree to optimize prioritization through compare-and-contrast decisions. Four factors to consider when assessing opportunities:
Opportunity sizing: "How many customers are affected and how often?" Remember: "all customers some of the time" is different from "some customers all of the time."
Market factors: competitive landscape & external trends make some opportunities table stakes (where missing it might lose a sale) and others strategic differentiators.
Company factors: What you decide to pursue needs to be coherent with your organizational context, at the macro (whole company) and micro (down to your team) level and in-between. Consider your strengths and weaknesses, available resources, strategic vision, etc.
Customer factors: not all opportunities are equally important to customers, even if we're successfully filtering for real customer needs or desires.
Weighing these factors will be messy and there is no exact formula for balancing tradeoffs and risks. Acknowledge confidence in what you know while leaving room for doubt. Lean into "two-way door decisions" which you can act on quickly and reverse if you find it's wrong. (We learn a LOT more by actually making the decision and experiencing outcomes rather than imagining outcomes.)
Anti-patterns to avoid: a) delaying decisions until there is more data (alluring, but not useful; only go deep enough to compare a set of siblings against each other); b) over-relying on one set of factors at the cost of others (it's a balance between the four factors); c) working backwards from the expected conclusion (keep an open mind).
Discovering Solutions
Chapter Eight: Supercharged Ideation
Creativity comes from a mix of three factors: a) the number of ideas we generate (fluency); b) how diverse the ideas are (flexibility); and c) how novel a given idea is (originality). For best results, have each person generate ideas independently prior to sharing as a group to spark new ideas. Tips:
"The most original ideas tend to come toward the end of the ideation session." Note that most solutions we put in our backlog are "first ideas," which is why it’s so important to focus on tackling opportunities in a systematic and holistic way to unlock creativity.
Find a rhythm that works for you, and empower yourself to take breaks to get unstuck.
Look to analogous products or industries (or even unrelated domains) for inspiration.
Consider your most extreme users: power users, first-time users, those in remote locations, etc.
Always anchor to your target opportunity first; go through the cycle of independent idea generation > group sharing as many times as you need until you have a broad set of ideas.
Use dot voting across your cross-functional team to narrow down to the most promising 3 ideas to test. If you have more than 3, have people quickly pitch the reason why they picked their ideas and then vote again (no need for extensive debate).
Risks of relying on collective brainstorming on its own: 1) we work harder when working alone (social loafing); 2) people in groups tend to create similar ideas (group conformity, e.g. they lack diversity); 3) we lose our train of thought while listening to others (production blocking); and 4) the performance of the group tends to be limited by the lowest-performing member (downward norm setting). To make it worse, group brainstorming creates an "illusion of group productivity" where they overestimate their performance.
Anti-patterns to avoid: a) not including diverse perspectives (go beyond the product trio); b) generating too many variations of the same idea (collapse those down and focus on real diversity); c) limiting ideation to only a single session (let people consider their ideas over time); and d) selecting ideas which don't address the target opportunity (don't abandon your opportunity solution tree).
Chapter Nine: Identifying Hidden Assumptions
Develop the skill of identifying the numerous assumptions which would need to be true for any given idea to succeed. This protects us from confirmation bias (where we ignore disconfirming evidence) and escalation of commitment (where we become more committed to an idea as we invest more). It’s useful to consider the important categories of assumptions relating to your product's:
Desirability - Will customers want it enough to do all of the things we need them to do?
Viability - Will this create a return for our business?
Feasibility - Is it possible for us to build it? (Not just technical feasibility)
Usability - Can a customer understand what to do?
Ethical implications - Will this create any potential harm?
Useful prompt to explore potential harm: "'If the New York Times/Wall Street Journal/BBC ran a front-page story about this solution that included your internal conversations about how the solution would work, what data you collected, how you used it, and how different players in the ecosystem benefited or didn’t, would that be a good thing? If not, why not?"
Story mapping is an excellent way to find assumptions. First, assume the solution already exists (so the map shows how users get value, not how you'll build it). Then, identify the key actors who need to interact to deliver value (sometimes your platform or interface may be an “actor” in this sense). Then, map out (sequentially) "the steps each actor has to take for anyone to get value from the solution," and be specific. Each step can then be broken down into the assumptions required for successful completion of the step, covering the 5 categories above. [The author provides an excellent illustrative example.]
Other tools for assumption identification include a) revisiting the nodes and linkages of your opportunity solution tree; and b) conducting a pre-mortem (remember to phrase the question as if the outcome is certain; e.g. try to understand why the product did fail, not why it might fail.)
Once you have your big list, identify the "leap of faith" assumptions to prioritize them for testing. The important part is how the assumptions rank relative to each other. (Don't get bogged down with discussion at this step.)
Anti-patterns to avoid: a) not generating enough assumptions (seek quantity to increase the likelihood the riskiest ones are in the list); b) phrasing assumptions such that you need them to be false (instead, it's easier if you need them to be true); c) not being specific enough (e.g. instead of "customers will have time," say "customers will take the time to browse all the options on our getting started page."); and d) favoring one category of assumptions at the cost of others (use the categories to catch your blind spots).
Chapter Ten: Testing Assumptions, Not Ideas
Early stage testing is best done with purpose-built tests for the riskiest assumptions alone. Save yourself the work of designing the whole idea and testing it all at once (and going through extensive iterations). "A strong assumption test simulates an experience, giving your participant the opportunity to behave either in accordance with your assumption or not. This behavior is what allows us to evaluate our assumption." Think carefully about what moment to simulate; you don't need to go any further! These will never be perfect, but it will help to mitigate risk in decisions being made. Further tips:
Test assumptions for 3 different ideas at the same time (they may have overlap). This helps prevent overcommitting to any one idea and enables triangulation with results.
Be specific about evaluation criteria: e.g. "At least 3 out of 10 people will do X." This gives you a chance to align around what signal gives your whole team confidence in a given test result in advance and reduces the risk of confirmation bias.
Start small and look for early signals before investing in large-scale experiments. Aim for tests that can be completed in a day or two. "Good tests kill flawed theories" (Karl Popper).
False positives and negatives are possible (and even likely) with rapid, small-scale assumption tests. Remember: a) proceed with skepticism and seek to uncover flaws in your tests; b) ultimately you get to triangulate between different assumption tests which all overlap in different ways; and c) you always have more solutions up your sleeve.
Remember: you're not discovering new universal truths about the universe, you're mitigating risk. So use a scientific mindset but don't dwell on scientific validity.
Lean on two tools in particular: 1) unmoderated user testing; and 2) one-question surveys. (Be careful who you recruit and how you word the questions; follow the rules of chapter 5.)
Further reading: a) UX for Lean Startups by Laura Klein; b) Testing Business Ideas by David Bland.
Anti-patterns to avoid: a) overly complex simulations (don't forget our intent to move quickly); b) using percentages when defining evaluation criteria (obscures the fact we're testing from small samples); c) failing to define enough evaluation criteria (pay attention to all of the things that need to happen for success); d) testing with the wrong audience; and e) designing for less than the best-case scenario. On the last note: "Design your assumption tests such that they are likely to pass [...] and with the most likely audience. You'll be surprised how often your assumption tests still fail."
Chapter Eleven: Measuring Impact
Even as we start to enjoy successful assumption tests, we need to stay focused on the end goal of driving the desired outcome. You don't need to measure everything in your early product; instead focus on instrumenting whatever you use as evaluation criteria for success. Additional tips:
Ask, "If one person did many actions, does this create as much value as many people doing one action?" If yes, count actions. If not, count people.
Even if the final outcome is hard to track, find some way to measure it. [Example of sending a follow-up email to ask users what came of someone’s job application submission.]
You don't need to test with the full solution to measure its potential impact on business outcomes. (A one-time pilot may help both sides vet the value of a potential partnership too.)
Remember: discovery & delivery work iteratively and support each other. More advanced discovery becomes indistinguishable from early delivery as you work with live prototypes and bigger groups of test users.
Anti-patterns to avoid: a) getting stuck trying to measure everything; b) focusing too much on assumption tests and forgetting to keep the solution aligned with the opportunity solution tree; c) forgetting to test (and verify) the connection between your product and business outcomes.
Chapter Twelve: Managing the Cycles
[Stories of real product teams using Continuous Discovery Habits]:
Example of Simply Business (large UK business-insurance providers) testing opportunities identified as a major headache for customers: late payments. Feedback from early tests showed customers did not want help from Simply Business with the issue. End result: opportunity not viable in the short-term, the team pivoted quickly to another opportunity.
Example of CarMax (large used car reseller) exploring customer value related to their reconditioning service. They discovered customers strongly valued vehicle-specific reconditioning information over generic information about the service. End result: defer the opportunity to later, prompt other teams to invest in the data pipeline to make it feasible.
Example of FCSAmerica (large agri-business creditor) exploring digital, self-services experiences to replace high-touch services. End result: iteratively worked toward a hybrid approach that retained the desired "trusted relationship" with their financial officer while bringing a lot of the data collection into the digital flow.
Example of Snagajob (hourly work marketplace) iterating through small opportunities around improvements to communication channels for hiring managers to more reliably connect with job seekers via text instead of via unreliable direct phone call, and then toward sending follow-up questions via survey, and then toward supporting with scheduling interviews, etc. End result: lots of small improvements which added up to a large impact.
Anti-patterns to avoid: a) overcommitting to an opportunity (don't be afraid to set something aside for now and move on to something more promising for your current objectives); b) avoiding hard opportunities (continuous discovery isn't just about short-term impact, it's the best way to tackle the big things that will take time to address adequately); c) drawing conclusions from shallow learnings (learn to lean into customer pushback when our initial ideas don't land well and understand why); or d) giving up before small changes have time to add up ("it may take a series of changes to move the needle on our outcome").
Chapter Thirteen: Show Your Work
Using the structure of Continuous Discovery is the most effective way to inform and engage stakeholders in the product process. This is critical, since lack of stakeholder buy-in may be enough to kill promising projects. When we lead with our conclusions - our roadmap, release plan, prioritized backlog - then stakeholders will have their own conclusions which will contend with ours, and we will not likely win the resulting opinion battle. Use the following sequence and prompts for successful co-creation with stakeholders (gather feedback for your consideration, and ONLY go as deep as relevant for the given stakeholder):
Set the scope of the conversation by aligning on the desired outcome at the top of your Opportunity Solution Tree. "Has anything changed since we last agreed to this outcome?"
Share your map of the opportunity space and check for gaps. "What might we have missed?"
Share decisions you've made so far in assessing and prioritizing the opportunities. "Would you have made a different decision?"
Share context around the target opportunity, including interview snapshots. "What questions do you have about the desire or pain point we intend to address?"
Share solutions you generated. "Do you have any ideas? Would you have chosen a different set to start with?" (ONLY do this once they’ve understood the context of the target opportunity.)
[for the top solutions] Then share story maps, assumption lists, assumption maps, and assumption test results, and gather feedback along the way.
"When we take the time to show our work, using visual artifacts like experience maps, opportunity solution trees, and story maps, we are inviting our stakeholders along for the journey with us. Instead of presenting our conclusion—this is the roadmap, release plan, and backlog that will help us reach our desired outcome—we are presenting the potential paths we might take to get there. We are inviting our stakeholders to help us choose the right path. Instead of presenting a conclusion, we are generating and evaluating options. This allows our stakeholders to be a part of the process. We are inviting them to co-create with us, which leads to much more buy-in and long-term success."
Anti-patterns to avoid: a) telling instead of showing (we have the curse of knowledge, so start at the beginning); b) overwhelming stakeholders with all the messy details (act as a smart filter, though you can always go deeper if needed); c) arguing with stakeholders about why their ideas won't work (don't shoot down their ideas, but use the opportunity solution tree to find where their suggestion would fit); d) trying to win the ideological battle instead of focusing on the decision at hand (choose your battles and be pragmatic when dealing with overzealous stakeholders; play the long-game).
PART 3: DEVELOPING YOUR CONTINUOUS DISCOVERY HABITS
Chapter Fourteen: Start Small, and Iterate
It's never impossible, too late, nor any amount of adoption too small to introduce elements of continuous discovery into your current process. Start now and iterate from there. Tips:
Remember the core driving force: get close to the customer.
Find your “product trio” of close cross-functional collaborator.s
The keystone habit (leading to numerous other benefits) is talking with customers weekly.
When tasked with delivering a specific solution, work backward into elements of discovery.
Reflect and improve your work through retrospectives. Introduce some discovery-related prompts, e.g. "What did we learn during this sprint that surprised us?" followed by "How could we have learned that sooner?"
Anti-patterns to avoid: a) jumping to a conclusion that some aspect of continuous discovery "would never work here" instead of focusing on what is within your control; b) becoming the annoying champion for the "right way" of working (don't let the perfect be the enemy of the good); c) waiting for permission instead of starting with what is within your control.
Chapter Fifteen: What’s Next?
[List of resources to continue learning about Continuous Discovery Processes.]
Read the full book (if you've only read this summary)
Product Talk monthly newsletter: ProductTalk.org
Continuous Discovery Habits membership community: Members.ProductTalk.org
Master Classes with the author: ProductTalk.org
Skills Deep-Dive Courses: Learn.ProductTalk.org
Hire a Product Talk Coach: teresa@producttalk.org
Thanks Evan for such a crisp and on-point summary, helped a lot. As someone just getting started with product and strategy, this was really insightful.