Data Landscape and Reports That Matter

Know where product evidence lives, what each source can answer and which reports support decisions. Keep account mappings and metric definitions consistent across every report and AI tool.

Why this matters

Enterprise answers cross telemetry, CRM, billing, support and other systems with different identifiers, refresh cycles and blind spots. Telemetry describes behavior, CRM records sales activity, billing records payment and support captures reported pain. Each provides part of the picture.

Own three foundations: a source inventory, a small catalog of decision-focused reports and governed metric definitions. They let people and AI tools use consistent evidence.

Chapter 05 covers instrumentation and experiments. Chapter 09 covers retention and health analysis. Here you build the landscape and governance. Chapter 19 covers connected access and verification.

The product data landscape

Ten sources, one map

Use these ten source types as the starting point for your inventory. The table is this book’s synthesis:

Triangulate source families. Telemetry and traces show behavior. Support, surveys and calls add explanations from a self-selected sample. CRM, billing and finance show commercial value under their recording conventions. Market sources reveal perceptions beyond your customer base.

The join key is the PM’s leverage

B2B questions often need joins. Adoption, payment and support trends require telemetry, billing and tickets linked at account level.

Account structures differ: CRM parents and subsidiaries, billing entities, telemetry tenants and support domains. One MSP can be one CRM account and sixty tenants. An unmanaged mapping can silently distort every cross-source report.

Record identifiers, mappings and known failures in the inventory. Assign an owner, since every cross-source report depends on them.

Every source is a data product

Zhamak Dehghani’s 2020 data-mesh principles recommend treating data as a product. Each domain owns data for others to discover, understand, trust and use securely.

Treat yourself as a customer of each source. Specify freshness, definitions and join-key requirements. Name the individual accountable for repairs.

Before you trust a number: the five-question check

Barr Moses of Monte Carlo uses the term data downtime for periods when data is partial, erroneous, missing or otherwise inaccurate. She describes five pillars of data health. Each converts naturally into a question a PM (or the PM’s harness) can ask before acting on a number:

1. Freshness. When was this data last updated, and is that later than the decision needs?
2. Quality. Are null rates, duplicate rates or out-of-range values unusual this period?
3. Volume. Did the row or event count jump or drop outside its normal range?
4. Schema. Did any field, event name or definition change recently?
5. Lineage. What upstream sources feed this number, and did any of them have a problem?

A tracking failure can resemble a behavior change. Run the five checks before explaining a metric movement.

Every report is a proxy

Jeff Bezos’s 2016 shareholder letter warns that processes, surveys and reports can replace attention to customers. Treat reports as proxies for value, quality and relationship health.

Goodhart’s observation, generalized by Marilyn Strathern, warns that targets can distort measures. An “accept” default might raise diagnosis acceptance without improving customer outcomes.

Keep verbatims, call excerpts and inspected traces beside metrics. Watch inputs that teams can game after they become targets.

The reports that matter

Inputs first: the weekly business review

Cedric Chin’s Commoncog account of Amazon’s Weekly Business Review (WBR) describes a process-control forum. It asks what customers experienced, how the business performed and whether targets remain on track.

Four features of the format, as Chin describes it, transfer directly to a B2B product team:

• Causal order. The deck starts with controllable input metrics (the levers teams can move this week), then the output metrics they’re meant to drive, then the financials. Teams report on outputs but work on inputs.
• Exception-driven discussion. Every metric is shown, but owners speak only to exceptional variation or a missed target. Normal weeks get “nothing to see here.” If an owner doesn’t know the cause, they say so, and a follow-up gets logged. Improvised explanations aren’t allowed.
• A fixed visual format. Chin describes the “6-12 graph”: the trailing six weeks on the left, the trailing twelve months on the right, last year as a faded line and targets marked. Because every chart looks the same, readers spot anomalies quickly.
• A living metric set. Metrics are added and dropped as their predictive power changes.

Chin reports that Amazon’s deck runs to 400 to 500 metrics, and that his own company’s version has 47. A product team needs far fewer. This book’s recommendation for a B2B product WBR is 15 to 30 metrics, each with a named owner. One caveat matters in enterprise: with hundreds rather than millions of customers, weekly percentages are noisy. Use trailing four-week views and show the account list behind any move, not just the percentage.

The twelve-report set

This book recommends a twelve-report catalog. Define each report by its questions, cadence, audience and decision:

Use the questions as acceptance criteria. Rebuild or retire reports that cannot answer them.

Notes on the reports that go wrong most often

Feature adoption. Pendo’s 2019 study of 615 subscriptions found 80% of features rarely or never used. Treat this one-platform, one-year vendor result as a warning. Show adoption breadth across accounts, depth within accounts and features with little use.

Retention and NRR. Net revenue retention looks like one number and is really a stack of choices. SaaS Capital’s 2025 benchmark defines it from a December-to-December snapshot of recurring revenue, counts upsell, cross-sell and price increases, and pairs it with gross revenue retention, which excludes expansion and so can’t exceed 100%. Another company might use trailing twelve months, or exclude price increases. Both are legitimate; mixing them isn’t. Chapter 09 covers the analysis. This report’s job is to state which definition it uses, every time.

Customer health. Gainsight’s guidance on health scores lists the usual inputs: usage against entitlements, feature adoption, sentiment, engagement, relationships and customer outcomes. Its most useful advice is the check: if churn keeps surprising you quarter after quarter, rework the score. A health report should show, at least quarterly, how well last quarter’s scores predicted this quarter’s renewals.

Pipeline influence. This is the report most easily inflated. “Influenced pipeline” usually means a capability was tagged on an opportunity or mentioned on a call. That’s correlation, not cause. Present it as evidence of where product shows up in deals, never as revenue the product “drove,” and always show lost deals alongside won ones. The most valuable rows are the gaps: capabilities that appear repeatedly in losses.

Pricing and discount. Apply Marn and Rosiello’s price waterfall: list price, negotiated discounts, credits, free months and realized unit price. Chapter 15 covers the strategy. This report locates leakage.

Support trends. Ranking themes by ticket count lets one noisy account set your priorities. Rank them twice: by volume and by the ARR of the accounts affected, and show which themes are new this period. Separate defects from how-to questions, since the first goes to engineering and the second to documentation and onboarding. Because the “other” category grows whenever the taxonomy drifts, ask the harness to read a sample of “other” tickets each month and propose new themes.

AI quality and cost. This report is new for most teams and is covered in the AI lens below.

QBR pack. An internal QBR reviews strategy, goals, outcomes and bets with executives. A customer QBR reviews account value: customer KPIs, ROI, prior goals, health and dated actions. Gainsight describes this account-review structure. Own the internal pack and govern definitions used by both. Chapter 29 covers the forums.

Portfolio review. This report compares product lines that measure themselves differently, so it only works when every line reports the same few metrics under the same definitions: ARR and growth, NRR, gross margin, cost to serve and the investment each line consumes. Show each line’s trend over at least four quarters beside its strategic role, because a line with low growth may be doing its job if its role is to protect the base. The decision it feeds is where to invest, maintain, harvest or divest, and Chapter 30 covers what happens after a retire decision.

One definition per metric

Why honest teams report different numbers

Benn Stancil identified the missing central metric repository in 2021. Definitions embedded separately in dashboards, spreadsheets and queries drift, producing different correctly computed answers.

Governance, at the level a product leader needs it, is the set of practices that stop this. It has four parts: written definitions, a single source of truth, a semantic layer that serves the definitions, and contracts with the owners of each source.

The metric dictionary

The metric dictionary is a written entry for every metric that appears in a recurring report. Chapter 17’s metric definition card pins down one metric for one question; the dictionary is the governed collection of those cards, extended with ownership, type, thresholds and versioning. This book’s template (in the artifacts section) asks for:

• a plain-language definition a customer could read;
• the formula in words: numerator, denominator, filters and exclusions;
• the grain and time window;
• the segments it may be cut by;
• the source of record and its name in the semantic layer;
• an owner who is accountable for it and a steward who maintains the logic;
• its type: input or output, North Star, health metric or guardrail;
• a target and a threshold for “exceptional variation”;
• known caveats and the decisions it informs;
• a version number and a change log.

The change log matters most. Definitions should change when the business changes, but never silently. When the definition of “active responder” changes in June, every chart that crosses June needs a marker, and anyone comparing this quarter to last needs to know.

A single source of truth

A single source of truth gives each metric one authoritative definition and computed value. Multiple reports read it rather than computing independent copies.

When two numbers disagree, the single source of truth also settles which one stands, so the meeting can spend its time on the business instead of on reconciliation.

The semantic layer as a concept

A semantic layer, also called a metrics layer, sits between raw data and consuming tools. Define measures, filters, grain and dimensions once. Dashboards and AI agents request the governed metric. As dbt’s documentation explains, definition changes then reach every consumer.

You don’t need to know how any particular semantic layer is built. You need to know three things. It exists or it doesn’t in your company. Your recurring reports should read from it wherever it does. And in 2026 it’s also the safest interface for an AI harness, a point the AI lens takes up. The industry is moving the same way: in September 2025 Snowflake announced the Open Semantic Interchange, an effort with dbt Labs, Salesforce, ThoughtSpot and others to create a vendor-neutral standard for semantic definitions, so that a metric means the same thing whether a BI tool or an AI agent asks for it.

Data contracts

A data contract is a written agreement between the owner of a dataset and the people who depend on it. The Open Data Contract Standard, maintained by the Bitol project, organizes a contract into sections that include schema, data quality expectations, service levels, team and roles, support and authoritative definitions. You won’t write the contract file, but you should ask for one, or at least a contract-shaped row in your inventory, for every source a recurring report depends on: who owns it, how fresh it will be, what quality checks run and who to call when it breaks.

How to apply it

Plan roughly six weeks of part-time work to establish or reset the data landscape and report set.

Weeks 1 and 2: Map what exists.

1. List the decisions first. Take the decision question bank from Chapter 17. For each recurring decision (quarterly planning, pricing changes, launch go or no-go, renewals at risk), note the questions it depends on.
2. Build the data source inventory. One row per source, using the inventory template. Ask your harness to draft it from whatever documentation, catalogs and dashboards it can reach, then verify each row with the owner in a 20-minute conversation. The questions for each owner: what’s the grain, how fresh is it, what’s the account identifier, what does everyone get wrong about this data, and who fixes it when it breaks?
3. Map the account key. For each source, record the account identifier and how it maps to the others. Mark every known break: partner-managed tenants, subsidiaries, merged accounts, trial tenants.
4. Inventory the existing reports. List every dashboard and recurring deck that mentions your product. For each, note who uses it and which decision it informs. Expect to find duplicates computing the same metric differently, and dashboards with no audience.

Weeks 3 and 4: Define and choose.

5. Pick the WBR metrics. Choose 15 to 30, ordered from controllable inputs to outputs to financials. Connect them to the North Star and metric tree from Chapter 02. Name an owner for each.
6. Write the metric dictionary entries. Start with the WBR metrics and the cross-functional ones (ARR, NRR, active accounts). Where two teams define a metric differently, don’t pick a winner in a spreadsheet. Get the owners in a room, agree on one definition, version it and log the change.
7. Draft the report catalog. For each of the twelve reports, write the questions it must answer, its cadence, audience, owner, sources and the decision it feeds. Mark which reports exist, which need rebuilding and which you’ll defer. Most teams should run five or six well before attempting all twelve.

Weeks 5 and 6: Build, run and prune.

8. Move definitions into the governed layer. Work with the data team so that each dictionary entry exists in the semantic layer (or the closest thing your company has), and point the recurring reports at it.
9. Run the WBR for four weeks before changing it. Use the exception-driven rules: owners speak only to exceptional variation, “I don’t know” is allowed and logged, and nobody improvises a cause. After four weeks, drop metrics that never triggered a discussion or a decision, and add the ones people kept asking about.
10. Retire what the catalog replaced. Archive duplicate dashboards, with a note pointing to the governed report. Leaving them up guarantees the old numbers resurface in someone’s deck.
11. Schedule the reviews. Put the weekly, monthly and quarterly reports on the calendar, with named owners, and add a quarterly review of the inventory and dictionary themselves. Sources change, owners move and definitions drift.

The inventory, dictionary and catalog are durable artifacts in your Product 360. The reports built on them are living artifacts, rebuilt every cycle.

The AI lens

When the product is AI-powered

AI features add a data source that most report sets ignore, and they change the economics enough that the report set has to follow.

AI traces are product data. A trace records each step of a request: the prompt and response, retrieval steps, tool calls, latency per step, tokens and cost, plus any quality scores attached afterward. That’s how the Langfuse documentation describes them. OpenTelemetry has published semantic conventions for generative AI that cover agent spans and MCP tool calls, along with conventions for specific model providers, so the vocabulary for traces is converging. (The conventions are still evolving; check their status before you depend on a particular field.) For an agent product, the trace is the only record of why the product did what it did. Put AI traces in your inventory with an owner, an access policy and a retention rule, because they routinely contain customer content.

Evals belong in the report set. Anthropic’s 2026 guide to evaluating agents distinguishes capability evals, which start low and measure progress on hard tasks, from regression evals, which should sit near 100% and catch anything that breaks. It also defines pass^k, the share of tasks where all k attempts succeed, for agents where consistency matters more than an occasional success. Those distinctions belong on the AI quality and cost report. A capability eval rising from 40% to 55% is good news; a regression suite falling from 99% to 94% is a release blocker.

Numbers alone won’t tell you what’s wrong. Hamel Husain and Shreya Shankar, in their evals FAQ, call error analysis the most important activity in evals: read real traces, write open notes on what went wrong, group the notes into a taxonomy of failures, and keep going until new traces stop revealing new failure types. They recommend that you personally annotate at least 30 traces before handing the work to an agent, and they favor simple pass or fail judgments over 1-to-5 ratings. The AI quality report should therefore include the current failure taxonomy and its counts, alongside the pass rates.

Cost becomes a product metric. Inference cost moves margin. The FinOps Foundation’s guidance on FinOps for AI names unit metrics that fit directly into a product report: cost per inference, cost per million tokens, cache hit rate, model fit (the share of requests sent to a more capable model than the task needs) and, most useful for a PM, cost per successful outcome: total run cost divided by the runs that met the quality bar. That last metric ties quality to cost. A cheaper model that halves cost per call but doubles failures can raise cost per successful outcome.

Outcome pricing turns a product metric into a billing unit. When you price per resolved task, per resolved ticket or per successful outcome, the definition of “resolved” is no longer only an analytics question. It’s on the invoice. Product, finance and billing must share one definition, versioned, with finance as a co-owner. Chapter 15 covers the pricing decision; the metric dictionary is where you make it auditable.

Doing the work with AI

Point connected AI tools at governed definitions. For example, dbt’s MCP server exposes metrics, lineage and freshness read-only. Chapter 19 covers access and connector mechanics.

Ask the governed layer, not the raw tables. Two studies show why. Spider 2.0, a benchmark of 632 real enterprise text-to-SQL tasks presented at ICLR 2025, found that the best agent tested solved 21.3% of them, against 91.2% on the older, simpler Spider 1.0 benchmark. A 2023 study by Juan Sequeda, Dean Allemang and Bryon Jacob found that GPT-4 answered 16% of questions correctly when querying an enterprise insurance database directly, and 54% when the same data was described through a knowledge graph that encoded the business meaning. Both studies used models that are now out of date, and the second came from a knowledge-graph vendor, so treat the exact numbers as historical. The direction is what matters: business context, encoded as definitions, makes AI answers on enterprise data far more reliable. Your metric dictionary and semantic layer are that context.

Make provenance part of every answer. Ask the harness to return, with every number, the metric name and definition version it used, the source, the data freshness and anything it excluded. Treat a number without provenance as a draft.

Run the trust check before the analysis. The five questions from earlier (freshness, quality, volume, schema, lineage) make a good preamble for any analysis prompt, and a harness can check them in seconds.

Ask for the denominator and the accounts. Percentages hide small numbers. In enterprise data, “acceptance fell 12 points” might be three accounts. Ask the harness to show the account list behind any move.

Spot-check one account by hand. For any number that will drive a decision, pick one account and trace it through each source with the owner. If the account doesn’t reconcile, the report doesn’t either.

Keep the anecdote attached. A harness can pull ticket verbatims, call excerpts and sample traces as easily as it pulls numbers. Ask for both. That’s how you resist the proxy problem Bezos described.

Two cautions. Connecting a harness to customer data, call recordings or traces is a data-handling decision, so use only approved tools and connectors. And the harness will confidently answer a question your data can’t answer; check the inventory’s “can’t tell you” column before you ask.

The Beacon case: one number, two definitions

The trigger. A draft keynote slide from marketing quotes a total for incidents Beacon has resolved since launch. The number is many times larger than the North Star trend in the product team’s weekly review. Both teams say they’re right, and both are. Marketing counted every incident Beacon correlated. The product team counts incidents resolved with Beacon’s help: incidents where a responder accepted Beacon’s diagnosis or an approved action ran. Neither definition was written down.

The inventory. You build the data source inventory over two weeks. Several rows matter more than you expected:

• Telemetry records alerts ingested, incidents correlated, diagnoses proposed and accepted, actions proposed, approved and run. The trap: test tenants and Northlight’s own internal tenant are mixed in with customers.
• CRM holds the 420 enterprise accounts plus partner accounts. The trap is Leo, the MSP operations manager. His 60 mid-market customers appear as child accounts under one partner account in the CRM but as 60 separate tenants in telemetry. Any per-account metric that ignores this mapping either drops them or counts them twice.
• Billing charges a platform fee per environment plus usage today. Once per-resolved-incident pricing launches, a “resolved incident” becomes a billing event. That makes the North Star definition a revenue definition too.
• Support tickets live in Northlight Desk, Northlight’s own ITSM product, which means the support data has an owner inside the company and a clean API.
• NPS goes to whoever the customer success team lists as the contact, usually Priya, the SRE lead. Marcus, the VP of IT Operations who signs the renewal, is rarely surveyed. The score describes the champion’s view, not the buyer’s.
• AI traces hold each diagnosis’s evidence chain, tool calls, tokens and cost. They also hold snippets of customer logs, which Sam, the security reviewer persona, will ask about.
• FP&A owns the margin numbers, but nobody has defined inference cost per resolved incident, which is the unit the new pricing will need.

The definition. You convene the owners of the North Star (you), billing (finance) and the keynote (product marketing) and agree one dictionary entry. “Incidents resolved with Beacon’s help” counts an incident once, even if it was correlated across several tools, when a responder accepted Beacon’s diagnosis, an approved Beacon action ran, or both. It excludes test and internal tenants. It’s measured weekly from the semantic layer, with finance as co-owner because it’s about to become a billing unit. Marketing gets a second, clearly labeled metric, “incidents correlated,” which it can use in the keynote with its own definition on the slide.

Three other entries need the same treatment. Diagnosis acceptance rate counts acceptances over diagnoses shown to a responder, not diagnoses generated, because many diagnoses are generated for incidents nobody opens. MTTR is reported as a median, from Beacon’s own incident timestamps, and labeled separately from any customer-reported MTTR. NRR is stated with its window and whether price changes count, so the 118% can be compared with next year’s.

The WBR. You design Beacon’s weekly review in causal order:

In the fourth week, most lines get “nothing to see here.” One doesn’t. Diagnosis acceptance has dropped for EU accounts over three weeks. The Diagnose PM doesn’t yet know why, says so, and the follow-up is logged. By the next review, error analysis on a sample of EU traces shows that a retrieval change is pulling evidence from the wrong region’s logs for customers with data-residency settings.

The AI quality and cost report. That finding connects to the incident everyone at Beacon remembers: the hallucinated root cause that led a customer to roll back the wrong service. When the team reread that trace, the agent had over-weighted the most recent deploy and never checked a database failover in another region. That case is now in the regression suite, along with the EU retrieval failure. The report tracks regression pass rate (target near 100%), production acceptance, rollbacks after a Beacon action, the failure taxonomy from error analysis and cost per successful diagnosis. The last metric explains the margin story better than total inference spend: it shows which failures are also costing money.

Pipeline influence. Joining CRM loss reasons with call transcripts shows a pattern in deals lost to prospects who don’t use Northlight Observe: buyers asking whether Beacon works with their existing observability tools. The sample is small and the link is correlational, but it bears directly on the build-on-Observe versus tool-agnostic tension, so it goes into the quarterly portfolio review.

The QBR packs. The customer QBR for Marcus now uses the governed definitions: MTTR trend, escalations avoided, incidents resolved with Beacon’s help, approval-gate statistics Sam can audit, and an ROI story for the CFO. The internal QBR puts Beacon beside Observe and Desk: 38% ARR growth, 118% NRR and a 72% gross margin that now has a unit-cost explanation and an owner.

What changed. Marketing labels the keynote count “incidents correlated.” Duplicate dashboards are archived with pointers. Six governed reports go live, with the remaining reports scheduled. Product, finance and billing sign the pricing definition. The work required an inventory, four metric definitions and accountable owners.

Product 360 artifacts

# Data source inventory · [Product] · v[n] · [Date]

**Owner:** [PM]   **Reviewed with:** [data, RevOps, finance, support, applied AI owners]   **Next review:** [Date, at least quarterly]

## Sources
| Source | System of record | Owner (named person and team) | Grain (event, account, opportunity, invoice, ticket, trace) | Freshness | Account identifier | Can tell you | Can't tell you | Known traps | Harness access (governed or raw; connector or MCP) | Contract or SLA? |
|---|---|---|---|---|---|---|---|---|---|---|
| Product telemetry | | | | | | | | | | |
| CRM | | | | | | | | | | |
| Billing and subscription | | | | | | | | | | |
| Support tickets | | | | | | | | | | |
| Surveys and NPS | | | | | | | | | | |
| Sales and customer calls | | | | | | | | | | |
| Product reviews and communities | | | | | | | | | | |
| Market and competitive data | | | | | | | | | | |
| Finance and FP&A | | | | | | | | | | |
| AI traces and evals | | | | | | | | | | |

## Account mapping
| Source | Identifier used | Maps to the master account by | Known breaks (partners, subsidiaries, test tenants, merges) | Who fixes it |
|---|---|---|---|---|
| | | | | |

## Sensitive data
| Source | Contains customer content or personal data? | Access policy | Retention | Approved tools for AI use |
|---|---|---|---|---|
| | | | | |

## Open requests to source owners
| Request (freshness, field, definition, join key) | Owner | Needed for which report | Due |
|---|---|---|---|
| | | | |

# Report catalog · [Product or portfolio] · [Date]

**Owner:** [Product leader]   **Rule:** every recurring report appears here, or it gets archived.

| # | Report | Questions it must answer | Decision it feeds | Cadence | Audience | Owner | Primary sources | Key metrics (dictionary IDs) | Status (live / rebuild / planned / deferred) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Weekly metrics review | What changed? Is it exceptional? Why? What are we doing? | Weekly priorities and escalations | Weekly | | | | | |
| 2 | Product health | Is the product working for customers? | Reliability investment, release holds | Daily view, weekly review | | | | | |
| 3 | Funnel and activation | Where do new accounts stall before first value? | Onboarding investment | Weekly, monthly deep dive | | | | | |
| 4 | Retention and NRR | Which cohorts expand, contract or churn, and why? | Retention bets, packaging | Monthly, quarterly | | | | | |
| 5 | Feature adoption | What's used, by whom, how broadly and deeply? What should we retire? | Invest, fix or retire | Monthly; 30, 60, 90 days after launch | | | | | |
| 6 | Customer health | Which accounts are at risk or ready to expand? | Account plans, escalations | Weekly list, monthly trend | | | | | |
| 7 | Pipeline influence | Which capabilities show up in won and lost deals? Which gaps block deals? | Roadmap, competitive response | Monthly, quarterly | | | | | |
| 8 | Pricing and discount | What do customers actually pay? Where does price leak? | Pricing and packaging changes | Monthly, quarterly | | | | | |
| 9 | Support trends | Top themes by volume and ARR affected; what's new? | Defect priorities, docs, deflection | Weekly digest, monthly | | | | | |
| 10 | AI quality and cost | Is the AI getting better or worse? What does a successful outcome cost? | Release gates, model choice, pricing | Weekly; every model or prompt change | | | | | |
| 11 | QBR pack (internal and customer) | Are bets on track? What value did this customer get? | Quarterly priorities; renewals | Quarterly | | | | | |
| 12 | Portfolio review | Where is investment against return across the portfolio? | Grow, sustain, harvest, retire | Quarterly or twice a year | | | | | |

## Archived reports
| Report | Replaced by | Archived on | Pointer left? |
|---|---|---|---|
| | | | |

# Weekly business review · [Product] · Week of [Date]

**Facilitator:** [name]   **Data as of:** [timestamp]   **Definitions:** metric dictionary v[n]   **Trust check:** passed / failed on [source]

## Rules
- Every metric is shown. Owners speak only to exceptional variation or a missed target.
- "Nothing to see here" is a complete answer for a normal week.
- "I don't know yet" is allowed. The follow-up gets logged with an owner and a date.
- No improvised causes.

## 1. Controllable inputs
| Metric (dictionary ID) | Owner | This week | Trailing 4 weeks | Same week last year | Target | Exception? | Comment |
|---|---|---|---|---|---|---|---|
| | | | | | | Y / N | |

## 2. Outputs
| Metric (dictionary ID) | Owner | This week | Trailing 4 weeks | Trailing 12 months | Target | Exception? | Comment |
|---|---|---|---|---|---|---|---|
| North Star: | | | | | | | |
| | | | | | | | |

## 3. Financials
| Metric | Owner | Latest | Plan | Variance | Exception? | Comment |
|---|---|---|---|---|---|---|
| | | | | | | |

## Accounts behind the moves
For each exception, list the accounts that drove it, with ARR and segment.

## Customer reality check
Two or three verbatims (tickets, calls, traces) that show what the numbers stand for this week.

## Follow-ups
| # | Question | Owner | Due | Answered? |
|---|---|---|---|---|
| | | | | |

## Metric set changes proposed
Add / drop / redefine, with reason. (Change only after four weeks of evidence.)

# QBR metrics pack · [Product] · Q[n] [Year]

**Version:** Internal (executive product review) / Customer ([Account name])   **Definitions:** metric dictionary v[n]   **Data as of:** [Date]

## A. Internal product QBR
### Summary
Three sentences: where we are against goals, the biggest change this quarter, the decision we need.

### Goals and bets
| Goal or bet | Target | Result | Status (on track / at risk / missed / killed) | Evidence |
|---|---|---|---|---|
| | | | | |

### Output metrics (quarter and trailing four quarters)
| Metric | Q-3 | Q-2 | Q-1 | This Q | Target | Comment |
|---|---|---|---|---|---|---|
| North Star | | | | | | |
| NRR (state window and price treatment) | | | | | | |
| GRR | | | | | | |
| ARR | | | | | | |
| Gross margin | | | | | | |
| AI cost per successful outcome (AI products) | | | | | | |

### What we learned
Top findings from adoption, support, pipeline and pricing reports.

### Decisions requested
1.
2.

## B. Customer QBR
### Executive summary
### The outcomes you bought us for
| Original buying reason | Metric | Baseline | This quarter | Trend |
|---|---|---|---|---|
| | | | | |

### Adoption and time to value
### ROI (method stated; customer's own numbers where possible)
### Progress against last quarter's goals
### Benchmark (only if the comparison group is defined)
### Health and risks
### Actions
| Action | Owner (ours / theirs) | Date |
|---|---|---|
| | | |

### Next QBR: [Date]

# Metric: [Name] · ID [metric-id] · v[n]

**Owner (accountable):** [name]   **Steward (maintains logic):** [name]   **Status:** Draft / Approved / Deprecated

## Definition
One sentence a customer could read.

## Formula in words
- Numerator:
- Denominator:
- Filters and exclusions (test tenants, internal accounts, trials):
- Deduplication rule:

## Grain and time
- Grain (event, user, account):
- Time window (weekly, trailing 12 months, snapshot date):
- Fiscal or calendar:

## Allowed segments
[Tier, region, channel, cohort, product line]

## Source
- Source of record:
- Semantic layer name:
- Freshness:

## Type
Input / Output · North Star / Health / Guardrail / Financial · Billing unit? Y / N (if Y, finance co-owns)

## Targets and thresholds
- Target:
- Exceptional variation means:

## Caveats
What this metric can't tell you, and how it could be gamed.

## Decisions it informs
[Reports and decisions]

## Change log
| Version | Date | What changed | Why | Approved by | Charts that need a marker |
|---|---|---|---|---|---|
| | | | | | |

# Data trust check · [Report or analysis] · [Date]

Run before any number drives a decision. Record failures before the answer, not after.

| Check | Question | Result | Action if failed |
|---|---|---|---|
| Freshness | When was each source last updated? Is that later than the decision needs? | | |
| Quality | Are null, duplicate or out-of-range rates unusual? | | |
| Volume | Did event or row counts move outside the normal range? | | |
| Schema | Did any field, event name or definition change in the last 30 days? | | |
| Lineage | Which upstream sources feed this number? Did any have a problem? | | |
| Definition | Which dictionary entry and version was used? | | |
| Denominator | What's the base, and how many accounts does it contain? | | |
| Hand check | Does one named account reconcile across sources? | | |

**Verdict:** Trust / Trust with caveats (list) / Don't use yet

Prompts

Goal: Get a first draft of the inventory to verify with source owners.
Paste in (or connect the harness to): data catalog entries, dashboard lists, data team documentation and any existing metric definitions. Use only approved tools and connectors.

Draft a data source inventory for [product]. Use one row per source: product telemetry, CRM, billing and subscription, support tickets, surveys and NPS, sales and customer calls, product reviews and communities, market and competitive data, finance and FP&A, AI traces and evals, plus any other source you find.

For each row give: system of record, owner (named team and person if documented), grain, refresh frequency, account identifier, what it can tell us, what it can't tell us, known data quality traps and how an AI assistant can reach it (governed layer or raw tables).

Rules:
- Cite the document each field came from. If nothing documents a field, write "Unknown: ask owner."
- List every place where account identifiers differ between sources.
- Name owners only where documented. Mark other owners "Unknown."

Check the output: take the draft to each owner and confirm every row in a short conversation. Count the "Unknown" cells; they're your request list.

Goal: Catch broken data before it becomes a decision.
Paste in: the question you're about to ask and the metric names involved.

Before answering the question below, check the data behind it and report on each item first:
1. Freshness: when was each source last refreshed?
2. Volume: did event or row counts for the last 7 days fall outside their usual range?
3. Schema: did any field, event name or metric definition change in the last 30 days?
4. Lineage: which upstream sources feed these metrics, and did any report problems?
5. Quality: are null, duplicate or out-of-range rates unusual?

Use governed metrics from the semantic layer where they exist, and name the definition version for each. If any check fails, stop and tell me what failed before giving any answer.

Question: [question]

Check the output: if the harness reports all checks passed without naming dates, counts or versions, ask it to show the evidence for each.

Goal: Pin one definition for a metric and expose where teams differ.
Paste in: every existing definition of the metric you can find (dashboards, decks, finance docs, contracts, pricing pages) with their sources.

Using the definitions below for [metric], do two things.

First, compare them. Make a table showing, for each source: numerator, denominator, filters and exclusions, time window, deduplication rule and who owns it. Highlight every difference that would change the number.

Second, draft one metric dictionary entry with: plain-language definition, formula in words, grain and time window, allowed segments, source of record, owner and steward, type (input or output; North Star, health, guardrail or financial; billing unit or not), caveats including how it could be gamed, and a change-log row explaining what this version changes from each existing definition.

Show conflicting definitions side by side. List the decisions their owners need to make.

[definitions with sources]

Check the output: make sure every difference in the table traces to a real source. The decisions list is the agenda for the meeting with the owners.

Goal: Produce an exception-driven WBR draft from governed metrics.
Paste in (or connect): the WBR metric list with dictionary IDs, owners and targets. Use an approved tool if the data includes customer names.

Prepare this week's business review for [product] using only the governed metrics listed. Keep the order: inputs, then outputs, then financials.

For each metric show this week, the trailing four weeks, the same week last year where available and the target. Flag a metric as exceptional only if it crosses its documented threshold. For each exception, list the accounts that drove the move, with ARR and segment.

For each exception, list questions for the owner and evidence needed to establish the cause.

Add two or three customer verbatims from this week's support tickets or call notes that relate to the exceptions, with sources.

State the data freshness and definition versions at the top.

Check the output: confirm the exception flags match the thresholds in the dictionary. Delete any sentence that offers a cause; that's the owner's job in the meeting.

Goal: See whether the AI is getting better or worse, and what each good outcome costs.
Paste in (or connect): eval results (capability and regression suites), a sample of production traces, user feedback events and AI cost data. Traces may contain customer content: use an approved tool only.

Build this period's AI quality and cost report for [product/feature].

1. Regression suite pass rate and capability suite pass rate, with the change from last period. Flag any regression below [threshold].
2. Production quality: [acceptance rate or equivalent], and user-reported failures.
3. Error analysis: from the sampled traces, group failures into categories using short, plain names. Give counts and one example trace ID for each. Mark categories that are new this period.
4. Cost: cost per request, cost per million tokens, cache hit rate and cost per successful outcome (total run cost divided by runs that met the quality bar). Note any requests routed to a more capable model than the task seemed to need.
5. The three changes most likely to improve cost per successful outcome.

Check the output: open at least five of the cited traces and confirm they show the failure named. If new failure categories appear, add a regression case for each.

Goal: Tell one customer the value story with governed numbers.
Paste in (or connect): the account's governed metrics, the original buying reasons from the CRM opportunity, last QBR's goals, open support tickets and the latest health score. Use an approved tool for customer data.

Draft a QBR metrics pack for [account] with these sections: executive summary (three sentences), the outcomes they bought us for with baseline and current values, adoption and time to value, ROI with the method stated, progress against last quarter's goals, health and risks, and proposed actions with owners on both sides.

Rules:
- Use only governed metrics and state the definition for each the first time it appears.
- Where ROI depends on an assumption (for example, an hourly cost), show the assumption and mark it for the customer to confirm.
- Include risks and all open escalations.

Check the output: reconcile two numbers by hand with the account team before the meeting. Remove any benchmark that doesn't define its comparison group.

Goal: Get evidence of product gaps in the pipeline without overstating influence.
Paste in (or connect): closed opportunities for the last two quarters with stated loss reasons, plus call transcript summaries or competitor mentions. Use an approved tool for call data.

For [product], analyze closed-won and closed-lost opportunities from [period].

1. List the capabilities and competitors mentioned, with counts in won and lost deals, by segment.
2. Highlight capabilities that appear much more often in losses than wins. For each, give three representative excerpts with opportunity IDs.
3. Compare stated loss reasons in the CRM with what the call excerpts suggest. Note where they disagree.
4. State the sample size for every claim, and label everything as correlation, not cause.

Report observed opportunity tags and mentions as correlations. Leave revenue-influence estimates to a separately defined analysis.

Check the output: read the excerpts behind the top two gaps yourself. If the counts are small, say so in the report.

Anti-patterns

Checklist

• A data source inventory covers all ten source types, with a named owner, grain, freshness, account identifier, “can tell you,” “can’t tell you” and known traps for each.
• The account mapping across telemetry, CRM, billing and support is written down, with known breaks (partners, subsidiaries, test tenants).
• Sources holding customer content or personal data, including AI traces, have an access policy, retention rule and approved tools listed.
• A report catalog lists every recurring report with its questions, decision, cadence, audience, owner and sources; duplicates are archived.
• The weekly business review runs in causal order (inputs, outputs, financials), is exception-driven and has a named owner for every metric.
• Every metric in a recurring report has a metric dictionary entry with an owner, a steward, a version and a change log.
• Cross-functional metrics (ARR, NRR, active accounts, any billing unit) have one agreed definition, with finance as co-owner where money is involved.
• Recurring reports and the AI harness read from the governed layer, not raw tables.
• The five-question trust check runs before any number drives a decision.
• For AI products, the AI quality and cost report tracks regression and capability evals, a failure taxonomy from error analysis and cost per successful outcome.
• Pipeline influence is reported as correlation, with sample sizes and lost deals shown.
• Internal and customer QBR packs use the same governed definitions.
• The inventory, dictionary and catalog are reviewed at least quarterly.