In The Loop—Chapter 13: Product Discovery vs. Product Management

In the fit systems enterprise, systems get things done. In the next seven chapters, we dive into the enterprise’s operating systems and meta systems.

Great companies become great because of extraordinary products. But where do great products come from? They emerge at the intersection of customer need and product capability. You choose the segments you will serve. You focus on unmet needs of target customers. You hypothesize and iterate ideas to discover product solutions that meet these needs. Then you build a product, deploy it, and leverage feedback to make it better.

It might feel like the work of one operating system, but I would argue products are built and managed within two distinct enterprise operating systems. Remember that the way you define system contours will shape how you lead, and how people in your company will think and act. As a systems thinker, you know that a system’s boundary has integrity when a preponderance of work occurs inside its edges, pursuing a unique purpose.The two product-oriented operating systems in the fit systems enterprise are product discovery and product management, each with a unique purpose:

• The product discovery system is the operating system that discovers and creates disruptive value breakthroughs on behalf of target customers
• The product management system is the operating system that makes sustaining improvements to current products on behalf of existing customers

The product management system interacts with another operating system:

• The revenue enginesystem proves product value by acquiring and retaining customers, thereby achieving revenue growth within the constraints of unit economics

The product discovery and product management systems are supported by three meta systems:

• The strategy / planning / architectures ystem guides the general direction of product development, and ensures sufficient resources to do the job
• The engineering system provides product teams the tools, agile methods, reference architectures, frameworks and infrastructure to build digital products that exhibit resilience, scalability and efficiency
• The DataOps system ensures product teams have the necessary data to track product value and drive continuous improvement

The revenue engine system impacts product management in three ways:

• The experience of prospects on the journey towards a purchase
• Brand identity
• Pricing

In my first book, Scaling the Revenue Engine, I discussed the power of creating a “sales ready product.” This is a product that has been designed to seamlessly move the prospect quickly from first product experience to initial pre-purchase value. The freemium model is one example of a sales ready product. In this model the prospect can use the product up to some usage threshold for free. Another variation is “free trial”. Sales ready products are designed to deliver a highly intuitive digital user experience to the pre-customer with rapid discovery of value. The premise is that if a prospect can find your product, test it and experience rapid value discovery before they buy, customer acquisition will be more successful.

The second connection point between the revenue engine system and the product management system is brand identity — as expressed in value proposition, competitive positioning and visual design. Brand identity must be expressed consistently, both within the product and in all external messaging.

The third connection point is pricing. The value you create and the value you capture (through pricing) are, of course, closely correlated.

A Story

My life changed the day in 2006 that a friend, the General Manager of Toyota Sunnyvale, approached me. “I’ve got problems in my internet department,” he said. “I think there might be a business opportunity.” After four months of discussions, we each wrote checks for $100,000 and ResponseLogix (later named Digital Air Strike) was born. I set up shop in Toyota Sunnyvale’s small second floor conference room. Thus began a six year journey of hardship, discovery, growth, good decisions and bad decisions that eventually yielded a moderately successful startup.

But I want to take you back to the beginning. At inception, we had a vague sense that auto dealer internet departments were screwing up how they handled prospect leads. Industry data showed that a third of all auto internet leads were not answered at all, and the average response time was over six hours. But why?

It was eye opening to sit right next to the ten-person internet department at Toyota Sunnyvale. Watching and learning, we began to see the problems. Prospect lead volume was lumpy. Some days were dead as doornails. Other days were super busy. On the busy days, time and again sales reps would be drawn away from their desks to serve customers arriving at the store. The CRM system could route leads to available reps, but whenever the volume of incoming leads exceeded available rep capacity, everything broke down. Back at his desk and confronting seven leads in the CRM queue, a rep would work from the most recent lead back, figuring (correctly) that the older leads would be less likely to close. Older leads routinely fell through the cracks, going entirely unanswered.

We learned that most auto dealer CRM systems could execute automatic responses to consumers, but they were boilerplate: “Thanks for your interest in Toyota Sunnyvale. A sales representative will reach out to you soon.” They failed to address the customer’s most common questions: “do you have ‘x’ in stock”, and “what’s the price.” We knew the consumer wasn’t seeking a quote from just one dealership. Many dealers were simultaneously in play. All else being equal, speed and transparency won out. The consumer wanted the price and availability of their vehicle of choice, and wanted to engage. We theorized that the first dealer to respond with an actual price quote on an actual vehicle would gain increased sales.

We quickly discovered that it was difficult to determine at any given moment the status of vehicles in stock (in all their make / model / trim permutations), the status of manufacturer incentives (which changed every month) and the proper price of a vehicle, given the dealer’s pricing rules. We’d need to access multiple separate systems inside a dealership — a complicated endeavor. We tried to avoid making it that difficult. We played around with all kinds of response template designs; they all did a better job of saying nothing. One day a top executive from Toyota happened to visit the store. Upon hearing about our efforts he cut right to the heart of it. “They want the price. Want to create something new? Get them the price.”

The guidance renewed our vigor to attack the technical difficulties. But we still needed proof that a rapid price quote would improve sales. We were caught in a dilemma: we didn’t have enough money to build a digital prototype, but we also couldn’t raise the money to build the software without some proof of concept.

So we decided as a first step to focus just on Toyota Sunnyvale, where we could directly observe the impact of everything we did. We recruited a small group of “quoters” and used the dealer CRM system to execute quotes. Every time a lead came in, we would manually access Toyota’s vehicle inventory system to check on availability. We would then check current manufacturer incentives and Toyota Sunnyvale’s pricing rules. Then we’d use the data to execute the quote in an email back to the customer, under the signature of the assigned Toyota Sunnyvale sales rep. The whole process took about 10 minutes per quote. It was messy, but to the consumer it looked like a custom response from a real sales rep.

On the first day we went live, sales reps reported that the quotes were changing the quality of customer phone conversations. Showing the price right away created trust. After three months, the data showed that Toyota Sunnyvale’s conversion rate from leads into appointments was rising. Soon after, we learned that the conversion rate from appointments to sales was also higher. We had our proof: a rapid, accurate price quote increases sales.

Proof in hand, we reached out to other dealerships and started to charge for our manual service. Four dealerships signed up. Soon sales were rising at these dealerships as well. Behind the curtain, of course, we were a chaotic, disorganized mess. Technical difficulties arose as we tried to remotely access and hack multiple auto dealer CRM and vehicle inventory systems. The duck looked like it was smiling, but under the water it was paddling like crazy just to stay in place.

Eventually, we had multiple proof points that rapid price quotes on in-stock vehicles could improve auto dealer sales. So we raised a seed round and built software to automate the quoting process. It was technically quite difficult — in fact, some of the technical challenges were never fully solved — but we did keep raising money, and we scaled over six years to about $20M in revenue and 2000 auto dealer customers. We used customer feedback to iterate on the product. Eventually our quotes featured not just the vehicle the consumer asked about, but relevant new and used alternatives. Sent “from” the assigned sales rep to the consumer within five minutes of receiving the lead, our quotes drove solid sales performance improvements for dealers. In time, we built out a diversified product line that addressed multiple stages of the dealer-to-consumer messaging continuum.

Looking back, I actually believe the steps we took to uncover MVP were the actions of a healthy product discovery system. We made many mistakes later in our scaling journey, but our first moves were spot on. By living inside an auto dealership for a year and a half, we immersed ourselves in our customer’s reality. We partnered with a visionary customer. That helped us to iterate our way towards a true value breakthrough much faster than would otherwise have been possible.

When you differentiate product discovery from product management, you turn on two spotlights. One shines on new product value breakthroughs. The other shines on existing product value optimizations. Two spotlights are better than one, especially for large enterprises.

The Product Discovery System

Market truth is in motion. An effective product discovery system is designed accordingly. At regular intervals in the scaling of a company, a new value breakthrough is needed. Once you achieve enterprise scale, your product discovery system should run continuously. By then, multiple innovation experiment teams should be active at all times. If it is run well, the product discovery system will deliver a slow drumbeat of value breakthroughs that propel growth. As these breakthroughs become mainstream products, the revenue growth offsets decline in older, more mature products.

At the outset of product discovery, any claim of customer-perceived value is just a hypothesis. There is no proof. Early on, the product discovery system must struggle through a high-variation phase where hypotheses are rapidly and iteratively tested, narrowing slowly towards truth. Only after deep customer immersion, hypothesis testing and rapid prototyping may clarity emerge. The product discovery system stands apart from the other systems because it operates from an extreme outside-in posture, independent of the “going concerns” of the enterprise. A new product initiative moves into the product management system and becomes part of the company’s growth engine once a value breakthrough has been achieved.

A healthy product discovery system is customer and human centric — uniquely focused on the needs and experience of all the people the product touches (often expressed as personas). Innovation experiment team members immerse themselves in the customer’s world and jobs to be done.

While the dominant feature of the product discovery system is wide variation and continuous iteration down a narrowing path, at a high level there are eight discrete steps in the system’s workflow:

• Immersion and ideation
• Epiphanies
• Hypothesis definition
• Hypothesis testing and validation
• Rapid prototyping
• Design validation
• Build and release initial product
• Complete minimum viable product

Here they are, with their stocks, flows and the high level feedback loops:

Including its people, workflows, technology and money flows, the product discovery system (for a single team) looks like this:

Of course, in a large enterprise multiple innovation teams are likely to be simultaneously at work seeking new product value breakthroughs. Each team is an independent domain team, like this:

In Chapter 7 I shared how small, autonomous, cross-functional teams can create superior system performance and increase system resilience. This is especially true in the product discovery system. The mission is a value breakthrough. In The Loop leaders make sure innovation projects are funded. They bring together the right teams, provide the teams business outcome objectives and step out of the way.

Start by grounding an innovation experiment team in the company’s mission, product vision and the broad team purpose — the business outcomes you hope to achieve. Keep the focus on business outcomes, not a tightly defined product solution. Expect periodic progress updates, but give the team space. Autonomy creates ownership. If the product discovery team is strong, it will discover the most important customer jobs to be done, ideate ways to do them better and iterate towards the right product solutions. The best product experiences and most thoughtful technical designs come from autonomous teams.

A critical success factor in this system is the pace of hypothesis testing. The faster a team can spin the hypothesize / test / iterate flywheel, the faster it will vector in on truth.

Let’s go through each step in the team’s workflow.

Immersion and Ideation

This first step is the most common failure point in an underperforming product discovery system. Too many companies seeking a value breakthrough underinvest in this step. Immerse! Live in the market, with the customer, with the problem. In my startup, we lived inside an auto dealership for eighteen months. The feedback loops are all around you. Get comfortable with ambiguity. Connect deeply and empathically with the humans occupying “target customer personas”. What jobs are they seeking to accomplish? What barriers stand in the way? How big a difference would it make to people and companies if the job to be done was made massively easier, or if it became fully automated?

As you immerse and explore, the jobs to be done will surface. You will begin to conceive of ways to make these jobs easier. At this stage, your mission is to create a wide array of alternative job-improving ideas. Through doing the job yourself, interviewing people who do the job, studying people doing the job and matching those experiences against what you know (in terms of technology capability, workflow insights, domain knowledge or whatever other relevant capabilities you bring), ideas will emerge. This creative process is often intuitive. You generate many ideas, and you expose them to the people in your chosen market and segments of interest. Their reactions cause you to iterate. One idea sparks the next. Almost all of your ideas “fail” or are “wrong,” but from each “failure” you learn something that helps you narrow incrementally towards truth.

Epiphanies

Somewhere along the way you begin to sense that you are closing in on something important. It may come in a flash of insight, or it may be a slow dawning conviction. These are epiphanies. An epiphany is pre-rational. It’s a “gut instinct.” But it’s an important step in the process. Great innovators don’t proceed without them. Many people searching for a value breakthrough seek to approach the entire process analytically, and as such they undervalue and distrust epiphanies. They prefer to live only in the world of data and science. But an epiphany isin the world of science. It is the starting point for a hypothesis, the first step in the scientific method. Trust your gut, and be open to your epiphanies.

Hypotheses

Epiphany in hand, you must now shift your mental mindset from pre-rational to rational. It’s time to express your epiphany as a hypothesis. In fact, as Steve Blank noted in his book “Four Steps to the Epiphany,” multiple hypotheses are required:

• Customer hypothesis — customer segmentation scheme, personas, jobs to be done, pain points for each persona, value of solving the pain
• Product hypothesis — how the product as you conceive it will improve the job to be done and solve the pain such that it delivers at least 10X improvement versus current solutions
• Market hypothesis — confirmation that there are enough relevant companies or consumers who conduct these jobs or experience these pain points that you could build a public company to serve them
• Competitor hypothesis — an assessment of the competitive set, and the degree to which the pain points to be solved are already being solved by existing companies as compared to your proposed product
• Pricing hypothesis — given the ROI or value that the product delivers, the estimated demand curve and proposed price points (including expansion pricing)
• Revenue engine hypothesis — given the expected LTV of the product, the profitable means by which you plan to acquire, retain and expand customers

Hypothesis Validation

Now you’re ready to return to the customer’s milieu. Every claim within the six hypotheses needs to be challenged and critically evaluated through deep customer engagement. The voice of the customer, as expressed in personas, is paramount. Here it is helpful to leverage both formal research and anecdotal insights. Test your hypothesis claims with as many target customers as you can. As you iterate on the hypothesis, see if you can recruit and select a customer feedback group that fits your defined target market. These should be folks willing to spend time with you on hypothesis refinement, prototype testing and review. This feedback group will help you further refine the six hypotheses to the point where you are ready to develop prototypes.

Rapid Prototyping

Having reacted to your initial hypotheses, the customer feedback group is now ready to critically review and comment on prototypes. As you create a prototype that matches the product hypothesis, garner reactions. Leverage feedback to create the next prototype. It’s not unheard of for teams to execute five prototypes per week, constantly iterating. Prototypes are often created and shared in paper form, or with light wireframes created within online tools. The job of the prototype is to convey the essential product features that address the customer’s job to be done as described in the product hypothesis, with the minimum possible design time. Keeping prototype creation fast and easy is key to frequent iteration. As the customer feedback group provides input, you will iterate the prototypes and their underlying hypotheses — hopefully on a narrowing path.

If your product is deeply technical, it’s different. Prototyping is focused on creating a product that can do the intended job. It can take months or years to solve hard technical problems. For these types of problems, your build / test / learn / iterate cycle will not involve human feedback so much as technical performance. Edison’s light bulb didn’t work until it did.

Design Validation

Once you have identified a prototype that appears to address the customer’s job to be done, it’s time to fill out the prototype — developing the visual design, the copy and content and the user experience workflows. As the prototype fills out, expand the customer feedback circle. If there were ten participants in your initial customer feedback group, find thirty or fifty more. The more reactions you receive, the higher quality the signal. Iterate, iterate, iterate. Then iterate again.

Initial Product Release

Initial product release (IPR) is achieved when you first make your platform or solution available to customers for use or purchase. For Facebook, it was the launch of its first website at Harvard. Its success at Harvard led Zuckerberg to expand it to Oxford and Stanford. This phase was the IPR phase.

Minimum Viable Product

An initial product release becomes a minimum viable product (MVP) when there is hard evidence of customer adoption and use. You have a small group of paying customers who value the core features of the product enough to keep paying. They all have long lists of improvement requests, but the core feature is viable enough to retain them. For Facebook, it was when Zuckerberg expanded to other schools, initially requiring users to have a .edu email address.

Final Thoughts on Product Discovery

How much should be invested in the product discovery system? The answer is, “it depends.” In the beginning of a startup, your investment in product discovery approaches one hundred percent of available capital. Once you have found product /market fit and your MVP moves into the product management system for further fixes, fills and finishing work, your investment in the product discovery system approaches zero. Not because continuing value improvements are no longer important (of course they are). But now the value improvements are of a type that occurs in the product management system. You are filling in product gaps, and the work will take up all your resources. The product discovery system returns to center stage at the value inflection point called Minimum Viable Expansion (MVE).

This is the point at which you add your next product, or expand to the next vertical. A true product expansion requires that you reactivate the product discovery system with its deep customer immersion, exploration of customer jobs to be done, the high variation of hypotheses and the slow, unpredictable walk down a narrowing path.

As you scale past Minimum Viable Expansion the product discovery system should operate continuously. It’s the only way to continuously reinvent yourself. At enterprise scale it would not be excessive to devote 5% of enterprise value to investment in product discovery (to fund innovation initiatives) and corporate development (to fund venture investments and acquisitions). If the enterprise’s core business is under significant duress, even more may be required.

For the fit systems enterprise, there are many ways to invest in product discovery. Here are a few:

• Google allows employees to spend 20% of their time working on side projects; side projects sparked the development of AdSense, Gmail, Google Now, and numerous autonomous vehicle projects
• Many enterprises initiate skunkworks projects inside the company; innovation teams are charged with product discovery and development operating within general boundaries
• Some enterprises challenge business units to devote a percentage of expense budgets to new product development
• Some enterprises have created “innovation experimentation teams” charged with new concept development, often co-located in accelerators or incubators like Plug & Play or GSV Labs — these teams have three months to develop a fundable concept (if funded, the entity is launched as a separate company with an enterprise company investment)

Unlike the other operating systems of the enterprise, product discovery efforts defy prediction. Teams may exist in a state of ambiguity and seeming lack of progress for sustained periods of time before finding signal and bursting forth towards MVP. As CEO, it’s on you to accept the product discovery system’s constraints and conform your expectations accordingly. Creating time-based goals and incentives for product discovery makes little sense. It’s better for you to focus on purpose, support the system through your own market and customer engagement, and ensure sufficient resources have been dedicated to the discovery of market truth.

Over the years, many a large enterprise has slid slowly into the abyss due to atrophy of its product discovery system. It scaled up for years on the back of some long-ago value breakthrough, operating solely within its product management system. But suddenly a bold new competitive innovation takes hold, and its value advantage evaporates. As the market shifts, it can’t — and it dies. If yours is a large enterprise, keep your product discovery system alive and healthy.

The Product Management System

With the product management system, a proven product already exists. You have visibility into the features that deliver the most value today, and where gaps may exist. This clarity helps you zone in more quickly on points of value optimization. You approach the fix, finish and fill work of product management with an outside-in, rapid prototyping approach.

The product management system exists to build out and improve the product for existing customers and prospects who look like them. Healthy product management systems create products customers love. This system operates on the principle of “narrow outside in.” Thesis variation is more narrow in product management than it is in product discovery. A segmentation scheme has been developed already, the ideal customer profile is clear and real data is flowing from an existing product. It’s different.

Product management is the system in charge of existing product enhancements, customer escalations and fixes, incremental optimizations, and upgrades for improved scalability, resilience, responsiveness, security and maintenance. The product management system advances both the generative imperative (through value optimization) and the adaptive imperative (by making the product more resilient, scalable, and efficient).

Notice how the product management system picks up where the product discovery system leaves off:

A healthy product management system starts with clarity of direction, segregation of duties and the balancing of alignment and autonomy. Here are some critical success factors:

• Have a clear product vision
• Embrace agile principles
• Determine the domains of your product
• Define the bounded purpose for each domain
• Establish OKRs that drive intended business outcomes for each domain
• Assign a team to each domain
• Get team composition right — product manager, designer, lead engineer, data analyst, user researcher, other engineers
• Challenge teams to deliver on business purpose through product ownership — they should own development, operational maintenance and scaling
• Ensure teams are effectively supported by the engineering and DataOps systems

Jay Melone, in a blog on Medium.com titled “Agile Sprints vs. Design Sprints”,¹ summarized well the workflow of product management. Melone teases out three factors — stage, criteria and timing. Timing will vary, of course. He emphasized that while it looks like a linear flow, it’s not. It’s highly iterative:

Inside each domain team, the system works as follows:

The product management system encircles within its boundaries the entire end-to-end product. In a large enterprise that means multiple domains and domain teams, which might look something like this:

Product management domain teams are data driven. They track feature usage and conversion data using time graphs to confirm progress against business outcome objectives, like this:

The choice of domain (how you split up product work) is an important architectural decision. Let’s go back to Spotify. Remember how the company split up its teams:

For Spotify, container squads focus on client platforms — iOS, Android, etc. The feature squads focus on features — such as product launch workflow, or search, or a segment of the user experience workflow. Teams need the technical architecture to be in alignment with the organizational architecture. Spotify leverages a microservices architecture to segregate small, loosely coupled services into domains, each of which is owned by a team.

Once again, the speed of the hypothesize / test / iterate flywheel is a critical success factor. But with the product management system you have an installed base of existing users. This enables your flywheel to be data driven. Invest in the speed of the flywheel. If a team in the product management system is not completing twenty A/B experiments a week on average, it is not performing up to potential.

Since we explored self-organized teams in Chapter 7, we won’t go deep again. But suffice it to say that teams need to be outcome focused, not output focused. If you are outcome focused, you focus on the impact of software on business outcomes. If you are output focused you build bloatware. You keep adding to an expanding mass of software that is rarely used but must be maintained. It sucks resources without adding value. Don’t build bloatware.

Each team’s mission is to build, design, commit, deploy and maintain software that advances an explicit, unique business purpose. OKRs can be used to define business outcome milestones (for instance, “Reduce time to launch from 30 days to three hours”).

Summary

Don’t begin working on feature additions and other product management tasks until you are sure you’ve nailed product discovery. Test hypotheses fast, and don’t feel badly when a test fails. Failure is a valuable lesson in what not to do. Make sure you have gathered enough data to be confident in its predictive power. Only once you know you have signal can you move decisively from product discovery into product management.

Do your teams generate outputs that match an output goal? Or do they yield outcomes that match a business outcome goal? The product discovery and product management systems are all about outcomes. For the fit systems enterprise, it’s always about outcomes.