Learning What to Build and Why

In Chapter 1, we looked at how strategy, discovery and development, deployment, and ongoing operations work together.

This chapter focuses on the discovery process, what it explores, what it helps you achieve, how to plan and conduct the work, which methods you can use, and how to translate what you learn into decisions and delivery work.

Discovery activities such as talking with people to understand their needs, exploring ideas and other research, helps you turn a fuzzy idea into a clear opportunity, figuring out if it makes sense to pursue and what it is you need to address.

You do this by investigating:

• People and stakeholders. Identifying who is affected, their values, needs, goals and priorities.
• Organizational goals and strategy. Understanding why the initiative matters, how it supports the organization's strategy or mission and how success will be measured.
• Processes and services. Understanding how work gets done and services are delivered by examining the work people and systems do as part of these processes.
• Information and data. Identifying what information is created, shared, stored, and used, where it comes from, who needs access to it, and how accurate, timely, secure, and private it needs to be.
• Technology and operations. Exploring the systems, platforms and technology environment that will enable or constrain your solution.
• Policies and governance impacts. Identifying any policies your solution needs to address.

You, your team, and stakeholders can make better decisions based on what you find. What you learn may strengthen your initial idea, challenge it, or lead you to rethink the problem, opportunity, or possible solutions. It helps you decide whether the opportunity is worth pursuing, clarify what needs to change and why, identify the most important needs and outcomes, and surface risks and constraints that may affect the work.

You can then translate what you learn to help you define the goals, requirements, and other needs that guide the development and implementation of your solution.

A Guide to the Discovery Process

Discovery involves repeatedly broadening your understanding and narrowing toward decisions as you learn more about the problem and possible responses. This helps you avoid committing to a solution before you understand the problem and encourages you to consider and evaluate alternatives so you can identify the most appropriate way to address the opportunity.

The UK Design Council's Double Diamond provides a useful way to visualize this broader pattern of divergent and convergent thinking where the two diamonds represent the problem and solution spaces and in each, you diverge by exploring broadly and then converge by narrowing toward a decision.

Fig 2.1 The Double Diamond shows how discovery alternates between exploring broadly and narrowing toward decisions in both the problem and solution spaces.

You start by exploring the problem broadly, talking with the people affected, reviewing available information and examining the wider context before narrowing toward a clearer definition of the problem or opportunity.

When exploring possible responses to these problems, you diverge again by considering multiple ideas and potential solutions. You then evaluate and refine these options before converging on an approach that aligns with organizational goals and stakeholder needs.

Dual Track Discovery and Development

Traditional models treat discovery as a single upfront phase, while many modern product teams treat it as a continuous loop running alongside development, deployment, and ongoing improvement.

In this dual track model, discovery stays slightly ahead of delivery by investigating needs, testing assumptions, refining requirements, and exploring options before the team commits to building them.

You start by conducting enough initial discovery to establish strategic direction, understand the main risks, determine whether the opportunity is worth pursuing, and prepare for the first stage of delivery. Depending on the initiative you're working on, this might involve process changes, a proof of concept, or an initial increment of working software or hardware. Discovery then continues as you learn more, helping you refine needs, requirements, priorities, and possible solutions as development progresses.

This reduces the risk of defining too much too early and allows your understanding and priorities to evolve as you, your team, and your stakeholders learn more.

Fig 2.2 In dual-track discovery, the team establishes an initial direction and identifies important requirements and risks, then continues learning and refining the backlog alongside delivery.

How Much Initial Discovery Is Enough?

Too little discovery can leave important requirements and risks poorly understood, while too much upfront specification can delay delivery and reduce flexibility (Jørgensen, 2024). Additionally, how much upfront discovery you need depends on the initiative's complexity, uncertainty, criticality, and risk. For example, a small change to a user interface may require limited initial discovery, while a system that monitors gas pipelines requires more, because errors could damage the environment and injure the people working on that pipeline.

Discovery effort and rigor typically increases when you have:

• Large organizational impact, where the solution affects many stakeholders, business processes, or dependent systems.
• Regulatory or compliance requirements, which may require formal documentation, review, and approval (Wiegers & Beatty, 2013).
• Technical uncertainty, such as implementing an unfamiliar technology or integrating with other systems.
• Environmental impact risk, where poor design or failure could cause unnecessary resource use, waste, or environmental harm.
• High safety criticality, where poor design or failure could cause physical harm, injury, or loss of life.

Fig 2.3 Higher initiative risk or uncertainty generally warrants greater initial discovery effort.

You have probably done enough initial discovery when you can explain the opportunity, intended outcomes, primary users and stakeholders, major constraints, critical risks, and the basis for the next investment or delivery decision.

Conducting Your Research and Analysis

Discovery involves exploring different questions and sources of data, so you need to be deliberate about where you focus your effort. Start by identifying what you need to understand, the assumptions you need to test, and the decisions that further evidence could help you make.

Wherever possible, through this work you should involve the people affected, as they can help you identify what matters, design and evaluate possible solutions, reducing the risk of implementing something that doesn't address their needs, or that of the organization.

Identify What You Know, Assume, and Need to Learn

At the start of discovery, you, your team, and your stakeholders will already know something about what you're exploring, bring assumptions based on prior experience and background, and have important questions you can't yet answer.

You can manage this mix of knowledge, assumptions, and uncertainty using a certainties, suppositions, and doubts (CSD) matrix to help you identify what you currently know, what you believe may be true, and what you still need to learn.

• Certainties are things you currently have evidence to support.
• Suppositions are assumptions or explanations that seem plausible but need to be tested.
• Doubts are questions or gaps in your knowledge you have that require further investigation.

To create your CSD matrix you:

1. Brainstorm individually, then discuss as a group. Ask participants to write their ideas on sticky notes and place them in the appropriate column.
2. Review and refine. Challenge whether what you identified as certainties are supported by evidence and identify which assumptions and doubts are most important to investigate.
3. Identify what you need to do. Use the suppositions and doubts columns to identify research activities, stakeholder interviews, workshops, observations, or data analysis that will help reduce uncertainty.

Once you have what you need to do you'll then need to decide which are important to investigate first.

Prioritize What You Need to Learn

At the start of discovery, you'll probably have more questions and assumptions than you can investigate at once. You cannot explore everything in equal depth, so focus first on the unknowns that could have the greatest effect on the success of the product or system.

Work with your team and stakeholders to identify the important questions and assumptions you need to investigate. Then consider each one based on:

• Importance. How significantly could it impact users, organizational outcomes, feasibility, cost, compliance, adoption, or other important aspects of the initiative.
• Uncertainty. How strong the evidence is and how confident you are that your current understanding is correct.

Questions and assumptions that are both important and uncertain should be a high priority because investigating them helps reduce risk.

Once you have identified the questions and assumptions that need attention, decide what you need to do to investigate them. For each high-priority item, consider:

• Decision or Question. What do you need to understand, decide or test.
• Evidence needed. The information needed to make a confident decision.
• Participants or sources. Who or what can provide that information.
• Method or activity. What research, analysis, and other activities could provide it.
• Owner and timeframe. Who will do the work, and when is it needed.

Match Methods to What You Need to Learn

Once you know what you need to investigate, choose methods that will help you gather the information or evidence you need. Different approaches answer different kinds of questions, so you'll often combine several during discovery.

For example, you might review documents to understand current policies and processes, analyze operational data to identify patterns or trends, then use interviews or observation to explore stakeholder needs, goals, and experiences. A workshop might then help you explore ideas and opportunities.

Using a variety of methods also lets you compare evidence from different sources and when these findings reinforce one another, you can have greater confidence in what you have learned and, if they conflict, may reveal gaps and areas you need to investigate further.

The following table can help you identify what you need to investigate and which methods may help.

These areas often overlap, so exploring one may uncover needs, issues, constraints, or opportunities in others.

Discovery Methods and Tools

The table below summarizes common discovery methods and the kinds of questions they can help you investigate. Use it as a guide when choosing methods based on what you need to understand and the evidence you need.

How Do You Know Discovery Is Working?

Discovery is working when what you learn helps you make better decisions about the opportunity and what to do next.

Signs that your discovery work is effective include:

• The team can explain why the work matters. Proposed capabilities and backlog items can be connected to organizational goals, stakeholder and user needs, evidence, risks, or constraints.
• Evidence is informing decisions. You can identify the information and evidence behind important findings, priorities, scope decisions, requirements, and possible solutions.
• You've identified risks and constraints. Important technical, organizational, operational, regulatory, environmental, and stakeholder risks and constraints are surfaced early enough to influence the work.
• You know what you still need to learn. The team understands which questions need to be answered before particular decisions are made and which can be explored through ongoing discovery.

Be mindful if your research mainly confirms decisions that have already been made, produces artifacts that are rarely used, or generates findings that have little effect on priorities. These may be signs that your discovery work is not going well.

Making Sense of What You Find

Discovery generates a lot of useful data and information such as interview and observation notes, workshop outputs, documents, data, photos, diagrams, prototypes, and findings from other research.

To make use of this you need to synthesize and make sense of it to identify what matters to your stakeholders, and turn it into insights that can inform decisions and the design of the solution. This usually involves:

1. Organizing information and evidence.
2. Identifying patterns, relationships, contradictions, and gaps.
3. Interpreting what those patterns mean and why they matter.
4. Translating the implications into decisions, needs, requirements, constraints, risks and initial design ideas.
5. Sharing these results with stakeholders for review and feedback.
6. Identifying questions that require further investigation.

The following approaches can help you work through this process and make the most of the data and information you've collected.

Affinity Mapping

Affinity mapping helps you organize your notes, observations, and other information you've collected into themes that can inform needs, goals, constraints, risks, and requirements.

Affinity mapping is particularly effective when done in a workshop with a broad mix of stakeholders such as departmental leads, role and user representatives, engineers, and support staff. These different perspectives help surface alternative interpretations of the research and generate more ideas for addressing needs and goals as they emerge.

How to Do It

1. Do some initial analysis. Analyze and synthesize your data before the workshop so the data you're working with is manageable.
2. Capture your observations. Write one observation per sticky note and link to the source such as an interview, observation, document review etc.
3. Cluster the notes into themes. Group notes into themes such as strategic goals, user needs, constraints, risks and potential functionality. Continue to cluster themes where you see relationships or overlaps.
4. Prioritize. Prioritize themes based on factors such as stakeholder and user impact, organizational goals, risk, constraints, feasibility, and the strength of the evidence.
5. Plan the next steps. Discuss, identify and plan any next steps such as where further research and investigation is required.
6. Document, share and validate. Share the draft themes and insights with stakeholders to confirm you captured and interpreted themes and other outcomes from the workshop accurately. This is also an opportunity for people to chime in on anything that was missed or add new ideas.

Moving from Themes to Requirements

To help you identify themes that impact the design of your solution and to start to define its requirements evaluate:

• What need does it surface or address?
• What does this require the system to do?
• What constraint does it create?
• What risk does it introduce?
• What assumption does it confirm or challenge?

Using AI and Machine Learning to Support Analysis

User feedback, support tickets and usage data can be valuable sources of information to help you identify user goals, system needs, pain points, and recurring problems. However this data can be challenging to review manually.

Machine learning and AI, including large language models, can support discovery by helping you classify, summarize, and analyze this material. For example, natural language processing can classify user-generated text into categories such as bug reports and feature requests (Maalej et al., 2016). Other techniques can help identify common themes, the features users discuss, and how they feel about those features through approaches such as topic modeling and sentiment analysis (Guzman & Maalej, 2014).

A simple workflow includes the following steps:

1. Define your question(s). Decide what you are trying to learn, such as where a process breaks down, what users struggle with, or what risks and constraints appear in the data.
2. Prepare the source material. Gather the notes, tickets, comments or documents. Remove names, identifiers, confidential information, or other data that should not be shared and use only tools approved by your organization.
3. Ask for a structured first pass. Give the LLM a clear task, such as identifying themes, grouping similar comments, summarizing pain points, or flagging possible needs, risks, and constraints.
4. Require evidence. Ask the LLM to connect each theme or finding to specific examples, quotations, observations, or other material you provided so you can verify how it reached its conclusions.
5. Review and compare. Check the LLM output against your own reading of sample material to help you identify any missing themes, unsupported claims, overgeneralizations, or places where important information may have been lost.
6. Identify themes and requirements. Compare AI-generated findings with your other research and translate what you find into needs statements, problem statements, risks, constraints, requirements, backlog items, or follow-up research questions.
7. Document your approach. Record what data was used, what tool supported the analysis, what prompts were used, who reviewed the output, and what limitations remain.

While these tools can help you make sense of this data you should treat their output as draft analysis and not final findings.

Using Your Discovery Findings

You'll need to share what you learned in ways that help your stakeholders make decisions, such as project sponsors who want a concise summary of the opportunity, its expected value and risks, and the development team and others involved in delivery, who want more details to guide their work.

The specific deliverables vary across organizations and initiatives, but most fall into several common types.

Describing the Problem or Opportunity

Early discovery often produces a clearer understanding of a problem that needs to be addressed or an opportunity the organization may want to pursue. A problem or opportunity statement provides a concise way to communicate this understanding and inform decisions about goals, scope, investment, and further discovery.

A problem statement summarizes what is happening, who is affected, why it matters, and the effects of the current situation based on the evidence available.

An opportunity statement describes a need, possibility, or potential area for improvement, who could benefit, and why pursuing it may create value. An opportunity might arise from changing user needs, new technology, changes in the market or regulatory environment, or possibilities identified through research and analysis.

These statements may be created as separate documents, short summaries, or incorporated into a larger opportunity brief.

Establishing the Rationale, Goals, Scope and High-Level Needs

Documents such as a Business Requirements Document, Product Requirements Document, or Vision and Scope Document help you explain why an initiative matters, what outcomes it should achieve, who it affects, and what boundaries, assumptions, risks, and constraints shape the solution.

These documents are especially useful when leaders need to make investment decisions, stakeholders need a high-level description of the proposed solution, or regulatory, contractual, and governance processes require formal review and approval.

Using a Backlog to Define, Refine and Guide Development of the Solution

As you progress and learn more you need to capture what you know in a way that informs the design and development of your solution.

The backlog is an effective tool for connecting what you learn through discovery to the design and development of the solution. It is an evolving, ordered list of what the team may need to learn, explore, build, change, or fix to achieve the goals of the solution.

It begins as an initial list of possible needs, questions, capabilities, and work. It becomes more detailed as you learn through research, design, and prototyping and refine your understanding of what is needed. New items are added as needs emerge. Existing items are clarified, refined, and reprioritized as the team learns more, while items that are no longer relevant are removed. In this way, the backlog becomes a shared and central record of what you know, what remains uncertain, which priorities have been agreed, and what work is planned.

In practice, many organizations use both formal documents and a backlog. A document such as a Vision and Scope Document communicates the initiative's vision, goals, and scope to decision-makers and other stakeholders, while the backlog guides the work required to deliver the solution.

In addition, if the solution involves service delivery or human–computer interaction, supplement these materials with design artifacts that help stakeholders visualize the proposed experience and interactions at a high level.

Tracing What's Built Back to What Was Learned

Whether you capture requirements in formal documents, the backlog, or both, you should be able to trace them from the goals and needs that led to them through to their implementation and deployment. This helps ensure that the features you develop serve a clear purpose and allows you to answer questions such as, "Why does this feature exist?" or "Do we have a feature that addresses the need we identified?"

Maintaining traceability also helps you assess the impact of changing requirements, reconsider work when its rationale changes, and provide the audit trail that may be required by leadership, contracts, regulations, or other governance processes.

In environments that require formal traceability, you may use a requirements traceability matrix to connect business requirements, detailed requirements, design elements, implementation work, and tests. Backlog and lifecycle management tools such as Jira and Azure DevOps can also support traceability by linking goals and requirements to backlog items and development work. This makes it easier to evaluate whether work still matters and check that development remains aligned with the larger purpose of the solution.

References