Responsible AI and Regulation

Responsible AI becomes concrete through disclosures, logs, approval gates, testing and customer evidence. The PM owns product requirements and supporting artifacts. Counsel owns legal interpretation. This chapter explains the product work, not legal advice.

Why this matters

AI questionnaires, production actions and regulation require evidence that a principles document cannot supply. Enterprise buyers ask about training data, injection defenses, oversight and emergency controls before purchasing.

The book’s late-2026 regulatory snapshot distinguishes existing transparency duties from later high-risk obligations. Logging, oversight and accuracy evidence take time to build. Confirm applicability and dates with counsel before relying on them.

Agents add tool misuse, privilege abuse and cascading failures, addressed in OWASP’s December 2025 Agentic Top 10. Translate significant risks into backlog requirements with owners and tests. Scale review effort to the use case’s risk, then retain the evidence in the Product 360.

The frameworks a PM actually needs

Principles become product requirements

The OECD’s AI Principles, adopted in 2019 and updated in May 2024, cover well-being, human rights, transparency, robustness and accountability. Microsoft emphasizes fairness, reliability, privacy, inclusiveness, transparency and accountability. NIST describes similar trust characteristics.

Translate shared principles into product questions, requirements and evidence. The six-principle mapping below is this book’s synthesis.

For each row, retain evidence that the specified control exists and works.

The EU AI Act in product terms

The EU AI Act (Regulation (EU) 2024/1689) entered into force in August 2024. In July 2026 the EU amended it with the Digital Omnibus on AI (Regulation (EU) 2026/1744). It reaches AI systems placed on the EU market or used in the EU, wherever the company is based. For a PM, three ideas carry most of the weight: roles, risk tiers and obligations by tier.

Roles. The Act assigns duties by role. The provider develops an AI system (or has one developed) and places it on the market under its own name. The deployer uses an AI system under its own authority. Importers, distributors and authorized representatives have smaller roles; providers of general-purpose AI (GPAI) models have a separate track. In a typical B2B stack, the model lab is the GPAI model provider, the SaaS company that builds a product on the model is the provider of the AI system, and the enterprise customer that uses the product is the deployer. A vendor that uses its own AI internally (say, to screen job applicants) is also a deployer.

Roles can switch. Under Article 25(1), a deployer, distributor or importer becomes the provider of a high-risk system if it puts its own name or trademark on it, substantially modifies it, or repurposes a system (including a general-purpose one) so that it becomes high-risk. Platform PMs feel this most: if customers can configure your agent to screen job candidates or score credit, they may turn themselves into providers, so your terms and documentation must state what the system is for and what it must not be used for.

Risk tiers. The Act sorts AI systems by intended use.

• Prohibited practices (Article 5) are banned outright. New prohibitions on generating non-consensual intimate imagery and child sexual abuse material apply from 2 December 2026.
• High-risk systems come in two kinds. Annex I covers AI in products already governed by the EU product safety laws that annex lists. Annex III lists eight use areas: biometrics; critical infrastructure; education; employment; essential services such as credit scoring and life and health insurance pricing; law enforcement; migration and border control; and justice and democratic processes.
• Transparency-only systems (Article 50) carry disclosure and marking duties.
• Everything else is minimal risk, with no specific duties beyond AI literacy.

Annex III has an exit. Under Article 6(3), a system in an Annex III area isn’t high-risk if it poses no significant risk and performs a narrow procedural task, improves the result of a completed human activity, detects patterns or deviations without replacing human assessment, or performs a preparatory task. The exit closes for profiling: an Annex III system that profiles people “shall always be considered to be high-risk” (Article 6(3)). A provider relying on the exit must document the assessment before placing the system on the market and register it (Article 6(4) and Article 49(2)). The Omnibus kept that registration.

Obligations by tier. For high-risk systems, Article 16 lists what providers must do: a risk management system, data governance, technical documentation, automatic logging, transparency and instructions for use for deployers, human oversight, and accuracy, robustness and cybersecurity, plus a quality management system, conformity assessment, an EU declaration of conformity, CE marking, registration and corrective action when something goes wrong.

Deployers have their own list in Article 26: follow the provider’s instructions, assign human oversight to competent people with authority to act, keep input data they control relevant and representative, monitor operation and suspend use if they see a risk, keep the logs under their control for at least six months, inform workers before workplace use and inform people when a high-risk system makes or supports decisions about them. Most of those duties depend on something the provider builds: logs, oversight controls and clear instructions.

Some deployers also owe a fundamental rights impact assessment (FRIA) under Article 27 before first use: public bodies, private entities providing public services, and deployers using AI for credit scoring or life and health insurance pricing. It covers the deployer’s process, frequency of use, affected people, specific risks, human oversight and what happens if harm occurs, and it complements any GDPR data protection impact assessment rather than duplicating it.

Transparency duties reach more systems, and they already apply. Under Article 50, providers must design systems that interact with people so that people know they’re dealing with AI, unless it’s obvious, and must mark synthetic audio, image, video and text in a machine-readable, detectable format where technically feasible. Deployers must inform people exposed to emotion recognition or biometric categorization, disclose deep fakes, and disclose AI-generated text published to inform the public on matters of public interest unless a human reviewed it and took editorial responsibility. A voluntary Code of Practice on marking and labelling AI-generated content, finalized in June 2026, gives providers and deployers a recognized way to meet these duties.

Serious incidents. Once the high-risk rules apply, Article 73 requires providers to report serious incidents to the market surveillance authority of the Member State where they occurred: no later than 15 days after becoming aware, no later than 10 days for a death, and no later than 2 days for a widespread infringement or a serious and irreversible disruption of critical infrastructure (Article 3(49)(b)). An incomplete first report is allowed. Deployers report to the provider first. If your product runs inside customers’ infrastructure operations, the two-day clock is the one to design your incident plan around.

Penalties. Under Article 99, prohibited practices can cost up to EUR 35 million or 7% of worldwide annual turnover, whichever is higher. Breaches of other operator obligations can cost up to EUR 15 million or 3%, and supplying incorrect information to authorities up to EUR 7.5 million or 1%. For SMEs and the small mid-caps the Omnibus added, the lower of the two figures applies.

These dates come from the consolidated text as amended by the Omnibus, as of October 2026. Confirm them with counsel before you commit a roadmap to them.

What to build in now. Most of what the high-risk rules require is what enterprise customers already ask for, so build it in even if none of your features is high-risk:

• An intended-purpose statement and a prohibited-use list in the product terms and documentation.
• Logging that lets a deployer keep at least six months of records.
• Human oversight controls: approval gates, overrides and a way to stop the system.
• Instructions for use written for the deployer, covering capabilities, limits and oversight.
• Accuracy and robustness metrics that come from your eval suite (Chapter 31).
• AI disclosure in the interface and machine-readable marking of generated content.
• A documented Article 6(3) assessment wherever a use case sits in an Annex III area and you believe it isn’t high-risk.

NIST AI RMF and the Generative AI Profile

The NIST AI Risk Management Framework (AI RMF 1.0, January 2023) is voluntary, and US enterprise and public sector buyers often ask vendors to align with it rather than with EU rules. Its core has four functions:

• Govern: the culture, policies, roles and accountability that run across everything else.
• Map: the context in which the system will be used, so risks can be framed.
• Measure: quantitative and qualitative analysis, evaluation and monitoring.
• Manage: prioritizing risks, allocating resources, responding, recovering and communicating.

Map defines intended use, people and harms. Measure supplies evals. Manage covers gates and incident response. Govern establishes accountability and the review process.

The Generative AI Profile (NIST AI 600-1, July 2024) tailors the framework to generative AI. It lists twelve risks that generative AI creates or worsens: CBRN information or capabilities; confabulation; dangerous, violent or hateful content; data privacy; environmental impacts; harmful bias or homogenization; human-AI configuration; information integrity; information security; intellectual property; obscene, degrading or abusive content; and value chain and component integration. It defines confabulation as “confidently stated but erroneous or false content,” a more precise term for what most teams call hallucination. The list works as a checklist for impact assessments and red-team scope.

NIST says the AI RMF is being revised under the July 2025 US AI Action Plan, with no date published, and in April 2026 it issued a concept note for a critical infrastructure profile. Check for a new version before you cite it in a customer document.

ISO/IEC 42001

ISO/IEC 42001:2023 is the first certifiable AI management system standard. Like ISO/IEC 27001 for information security, it requires policies, roles, assessments, controls, internal audit and improvement. Enterprise questionnaires increasingly ask about it.

Security or compliance usually leads certification. PMs supply product impact assessments, approved lifecycle changes and supplier controls. Keep artifacts reusable across product reviews and management-system audits.

The US: states move, Washington pushes back

The US has no comprehensive federal AI law. The states are legislating, and in 2026 the federal government is pushing back.

• Colorado enacted SB24-205, a broad law on algorithmic discrimination by high-risk AI systems, in May 2024. A 2025 special session (SB25B-004) delayed its duties to 30 June 2026. That date has passed, so before you plan around the law, confirm with counsel whether those duties now apply as written or were changed again in the 2026 session.
• California’s SB 53, the Transparency in Frontier Artificial Intelligence Act, signed in September 2025, requires large frontier model developers to publish a safety framework and report critical safety incidents. It targets model developers rather than most B2B application vendors, though your model provider’s disclosures under it can feed your own documentation.
• Texas enacted HB 149, the Texas Responsible AI Governance Act, in June 2025, effective 1 January 2026. It regulates the use of AI systems in the state and carries civil penalties.

At the federal level, Executive Order 14365, signed on 11 December 2025, aims at a national policy framework and at challenging state AI laws the administration considers burdensome; it names Colorado’s law. It directed a Department of Justice AI Litigation Task Force, a Commerce Department review of state laws, an FTC policy statement on AI and a recommendation for federal legislation that would preempt state laws, with carve-outs such as child safety. Check what those directives have produced by the time you read this.

For US sales, maintain a dated register of laws customers raise alongside contractual and procurement requirements. Use NIST AI RMF to structure risk work and have counsel interpret applicable law.

Other jurisdictions

Two more matter if you sell in Asia. South Korea’s Framework Act on AI took effect on 22 January 2026, with duties for “high-impact” AI (risk management, explanation, user protection, human oversight, records), notice and labelling for generative AI, and a domestic representative for foreign operators above certain thresholds. China’s Measures for Labeling AI-Generated Synthetic Content took effect on 1 September 2025 and require both visible labels and metadata labels. The common thread with EU Article 50 is labelling: design one content-marking capability that can satisfy several regimes.

Data protection: GDPR and the DPIA

AI regulation sits on top of data protection law. Under GDPR Article 35, a data protection impact assessment (DPIA) is required where processing is likely to result in a high risk to people, and always for systematic and extensive evaluation of people (including profiling) with legal or similarly significant effects, large-scale processing of special-category or criminal data, and large-scale systematic monitoring of public areas. A DPIA must describe the processing and its purposes, assess necessity and proportionality, assess the risks to people and set out the measures to address them.

If your company trains or fine-tunes models on personal data, your privacy team will also work from the European Data Protection Board’s Opinion 28/2024 (December 2024) on GDPR and AI models.

In most B2B products the customer is the data controller and you’re the processor. Three commitments then sit at the center of every enterprise deal: whether you train on customer data, where data is processed and stored, and which sub-processors (including model providers) touch it. State them once, consistently, in the data processing agreement and the system card.

The evidence a PM owns

The evidence behind the frameworks is a small set of documents and processes, most of which the PM drafts or coordinates.

AI impact assessments. Document intended use, affected people, harms and mitigations before development or release. Microsoft publishes an assessment template, ISO/IEC 42001 expects an AI system assessment, and some EU deployers need a FRIA. GDPR may require a DPIA for high-risk personal-data processing. Answer shared questions once per use case, cross-reference specialist assessments and retain each specialist’s contribution.

Model cards and system cards. Margaret Mitchell and colleagues’ 2019 model-card proposal records intended and unsuitable uses, evaluation methods and performance across groups and intersections. Model labs also publish system cards. Anthropic’s Transparency Hub, for example, links safety, injection-robustness and external red-team reports.

Your provider owns its model card. You own the product system card: models and purposes, data flows, tools, permissions, autonomy, eval results, limits, monitoring and user disclosures. Use it as the source for questionnaires and deployer instructions.

Red-teaming. NIST AI 600-1 describes structured testing for inaccurate, harmful or discriminatory outputs. It distinguishes public, expert, combined and human-AI exercises. Include deployment expertise and analyze findings before using them to make decisions.

Use relevant categories from OWASP’s LLM Applications 2025 Top 10 and Agentic Applications Top 10 (December 2025). The first covers injection, disclosure, output handling and excessive agency. The second adds goal hijack, tool misuse, privilege abuse, memory poisoning, cascading failures and rogue agents. Define scenarios and failure criteria, then add findings to evals.

AI incident response. Extend security response to wrong outputs at scale, unsafe actions, injected instructions, privacy leaks, fairness failures and silent model changes. Provide controls to disable actions while retaining suggestions and to roll back prompts or models. Confirm regulatory clocks, GDPR obligations and contract notices with specialists. NIST AI 600-1 recommends reviewing generative-AI gaps. The PM coordinates severity definitions, customer communication and new eval cases.

Customer AI security questionnaires. The Cloud Security Alliance’s July 2025 AI Controls Matrix has 243 objectives across 18 domains, mapped to ISO/IEC 42001, NIST and the EU AI Act. Its AI-CAIQ questionnaire provides an answer-bank structure. Store one canonical answer per question with evidence, owner and review date, then map customer wording to it.

Responsible AI review inside a product org

Review works when it’s proportional, early and attached to the product’s lifecycle. This book recommends the following pattern, which fits NIST’s Govern function and the ISO/IEC 42001 management-system approach:

• Triage at the idea stage. Every AI use case gets a quick risk tier. Most land in “light review,” and the team moves on.
• Gates where decisions happen. Review at PRD sign-off, before beta, before general availability and on any material change: a new model, a new tool permission, a new data source, a new market or a new autonomy level.
• Clear roles. The PM owns the impact assessment. Applied AI owns evals. Security owns the threat model. Legal and privacy advise. One named approver can block, and that person isn’t the PM.
• Evidence with the product. Records live in the product’s Product 360, where the next reviewer, auditor or customer can find them.

How to apply it

This method brings one AI product, or one major AI feature, to a defensible state. Run it with your applied AI lead, a security partner and counsel. Plan on four to six weeks for the first pass; after that it’s maintenance.

1. Inventory the AI use cases. List every place the product uses a model; a summarizer, a classifier and an agent that acts are three use cases with three risk profiles. Note each one’s model and provider, the data it sees and what it can change.
2. Triage each use case. Record your role, screen against the prohibited list and the Annex III areas, decide whether transparency duties apply and assign an internal risk tier. Where a use case sits near an Annex III area, write down the reasoning and the Article 6(3) conditions you think apply, and let counsel decide.
3. Write the impact assessment. For anything above the lowest tier, describe intended purpose and prohibited uses, affected people, harms by principle and the twelve NIST generative AI risks, and the mitigations. Ask privacy whether a DPIA is needed, and cross-reference it.
4. Turn harms into requirements. Each significant harm becomes a product requirement with an acceptance test: an eval slice, an approval gate, a disclosure, a log field, a permission scope. Put them in the AI PRD (Chapter 31) and the backlog, not only in the assessment.
5. Red-team before beta. Scope the plan from the impact assessment and the OWASP lists, use at least one expert and one AI-driven approach, and convert every confirmed finding into an eval case and a backlog item with a severity.
6. Write the system card and instructions for use. Draft them from engineering sources and eval results. Have engineering confirm every technical claim. Publish a customer-facing version.
7. Prepare the incident plan. Define AI-specific severities, the kill switches available for each use case, who decides, who tells customers and what the regulatory and contractual clocks are. Run one tabletop exercise before general availability.
8. Load the answer bank. Map your system card, assessment and controls to the AI-CAIQ structure. Each answer gets an owner and a review date.
9. Pass the review gate. Present the evidence to the named approver. Record the decision, the residual risks accepted and who accepted them in the decision log (Chapter 29).
10. Monitor and re-review. Watch eval results on production samples, user overrides, escalations and incidents. Any material change sends the use case back to step 3. Recheck the regulatory dates and state laws quarterly.

The AI lens

Building AI-powered products

Some questions only arise when the product itself is AI.

• Agents change the threat model. When an agent reads logs, tickets or web pages, any text it reads can carry instructions. That’s the OWASP agent goal hijack risk, and it means every connector is an attack surface. Scope each tool to the least privilege it needs, treat retrieved content as data rather than instructions and put an approval gate in front of actions that are hard to reverse. Chapter 31’s autonomy dial is a safety control as well as a UX choice.
• Connectors and MCP servers are suppliers. A third-party MCP server your agent calls is part of your supply chain. List each one in the system card with its maintainer and what it can access.
• Model changes are material changes. A provider’s model update can shift behavior without any change to your code. Treat it like a release: rerun regression evals, update the system card and re-review if the risk profile moved.
• Disclosure is a design problem. Article 50 requires people to know they’re interacting with AI unless it’s obvious. In an enterprise tool, the more useful test is whether a responder can tell which parts of a screen the agent wrote and what evidence it used, which is also the core of good trust UX.
• Your provider’s documents are your inputs. GPAI model providers have had EU obligations since August 2025, and the GPAI Code of Practice includes a Model Documentation Form. Ask your provider for it and for its system cards.

Doing the work with AI

An approved harness can draft assessments, system cards, scenarios and questionnaire answers from PRDs, evals, tickets and policies. Use it to identify claims unsupported by the supplied evidence.

Three rules keep this safe.

• The model drafts; a named human signs. Every questionnaire answer, assessment and system card has an owner who checks it against evidence. An invented answer misrepresents your product to a customer, whoever typed it.
• Never let the model make the legal call. A model can lay out the Article 6(3) conditions and argue both sides. Whether your use case is high-risk is for counsel.
• Use approved tools for confidential material. Questionnaires, contracts and incident details belong only in the harness your company has approved for that data, with connectors scoped to the task.

Supply current official sources for regulatory work. Check every citation and distinguish the book’s dated snapshot from the rules applicable to a customer today.

The Beacon case: getting Beacon Agents ready for an EU bank

Triage. You triage each Beacon capability separately, because their risks differ.

Northlight provides Beacon; the bank deploys it; the model lab provides the GPAI model. The open classification question concerns critical digital infrastructure. You document the facts, possible Article 6(3) reasoning for Diagnose and the stronger concerns around Act, then ask counsel to decide. Meanwhile, Act gets six-month log-retention capability, oversight controls, deployer instructions and eval evidence. Beacon does not evaluate people, so the case does not invoke the employment or essential-services categories.

One more role question: Leo’s MSP wants to offer Beacon under its own brand to its sixty mid-market customers. Under Article 25(1), putting its name on a high-risk system would make Leo’s firm the provider. That matters only if a use becomes high-risk, but you add it to the partner terms.

Impact assessment and requirements. Act’s assessment covers responders and people affected by production changes. Confabulation, security, over-reliance and component integration become testable requirements. Show evidence beside rollback approvals and log the action, source, approver and model version. Keep connectors read-only except for explicit action scopes. Privacy confirms processor status, default exclusion of customer-data training and a DPIA support pack for customers; logs can contain usernames and IP addresses.

Red team. Before the Northlight Live beta, the applied AI engineers and an external security firm run a two-week red team: instructions hidden in log lines and tickets (goal hijack), a malicious MCP server (agentic supply chain), a runbook step used to widen access (privilege abuse), one rollback triggering others (cascading failures) and a confident wrong root cause on a multi-region incident. A model generates hundreds of variants of the injection attacks. Two findings are severe: an instruction in a change-ticket description could steer the agent’s proposed action, and the agent could chain two allowlisted steps into an action no customer had allowlisted. Both get fixes and eval cases, and the beta waits a week.

System card and questionnaire. The harness drafts from designs, evals and connector inventory. Engineers correct four claims, including retention. The card records models, permissions, approvals, autonomy, diagnosis acceptance, multi-region limits, logs and disclosure. Using the AI-CAIQ answer bank, the 140-question response takes three days rather than three weeks in this fictional case. Owners sign each answer. Eleven unmatched questions receive new human-written entries.

The incident replay and the review. A tabletop replay reduces severity agreement from forty minutes to five. A per-customer kill switch disables proposed and autonomous actions while diagnosis continues. At pre-GA review, the head of security records approval with two residual risks: multi-region quality and the classification question awaiting counsel.

What it changed. The bank renews in this fictional example. The review produced least-privilege access, audit logs, eval evidence and an approval gate that also support future sales.

Product 360 artifacts

These artifacts extend the AI risk assessment in Chapter 31. Use the triage for every AI use case; use the rest in proportion to the tier. None of them replaces advice from counsel.

# AI use-case risk triage · [Product] · [Use case] · [Date]

**Owner:** [PM]   **Reviewed by:** [legal / privacy / security]   **Next review:** [Date or trigger]

## The use case
- What it does, in one sentence:
- Who uses it, and who is affected by its output:
- Model(s) and provider(s):
- Data it reads (including any personal data):
- What it can change (none / suggests only / acts with approval / acts autonomously):

## Role
- Our EU AI Act role: provider / deployer / both
- Could a customer or partner become a provider (own branding, substantial modification, new high-risk purpose)? [yes/no, how]

## Screens
| Screen | Answer | Reasoning |
|---|---|---|
| Any prohibited practice (Art. 5)? | | |
| In an Annex III area? Which one? | | |
| Profiles people? (if yes and Annex III: high-risk) | | |
| Article 6(3) conditions claimed, if any | | |
| Annex I regulated product? | | |
| Interacts with people or generates content (Art. 50)? | | |
| Personal data: DPIA trigger likely? | | |
| Other jurisdictions in scope (US states, Korea, China, others) | | |

## Internal tier
Low / Medium / High, because [reason]. Review track: light / standard / full board.

## Open questions for counsel
1.

## Decision
[Proceed / proceed with conditions / stop], decided by [name] on [date].

# AI impact assessment · [Product] · [Use case] · v[n] · [Date]

**Owner:** [PM]   **Contributors:** [applied AI, security, privacy, legal, UX]   **Linked:** triage, DPIA [link or n/a], FRIA support [link or n/a]

## 1. Purpose and boundaries
- Intended purpose:
- Intended users and settings:
- Prohibited and out-of-scope uses (as they will appear in terms and docs):

## 2. Affected people
| Group | How they are affected | Can they see or contest the output? |
|---|---|---|
| | | |

## 3. Harms
| # | Harm | Principle or NIST GAI risk | Who is harmed | Likelihood | Severity | Current evidence |
|---|---|---|---|---|---|---|
| 1 | | | | | | |

## 4. Mitigations as requirements
| Harm # | Requirement | Acceptance test (eval, gate, log field, UX) | Backlog item | Owner |
|---|---|---|---|---|
| | | | | |

## 5. Human oversight
- Who reviews or approves, at which step, with what information:
- How a user overrides or stops the system:

## 6. Data
- Personal data involved, retention, residency, sub-processors:
- Training on customer data: yes / no; if yes, basis and opt-out:

## 7. Fairness and performance
- Slices evaluated (segment, language, region, other):
- Results and gaps:

## 8. Residual risks
| Risk | Why accepted | Accepted by | Revisit by |
|---|---|---|---|
| | | | |

## 9. Change triggers
Re-assess on: new model or version, new tool permission, new data source, new market, new autonomy level, or a severity 1 or 2 incident.

# System card · [Product or feature] · v[n] · [Date]

Audience: customer security, compliance and admins. Every claim must be confirmed by its engineering owner.

## Overview
- What the system does and for whom:
- Intended purpose and prohibited uses:
- Our role and the customer's role (EU AI Act):

## Models
| Use | Model and version | Provider | Hosting region | Provider documentation |
|---|---|---|---|---|
| | | | | |

## Data flows
- Data the system reads, from where (connectors, MCP servers):
- What is stored, where, for how long:
- Training on customer data: [default and options]
- Sub-processors:

## Tools, permissions and autonomy
| Action class | Permission scope | Autonomy level (suggest / approve / autonomous) | Who can change it |
|---|---|---|---|
| | | | |

## Evaluation
- What we test, how often, and on what data:
- Current results (headline metrics and slices):
- Red-team scope and date of last exercise:

## Known limitations
-

## Human oversight and controls
- Approval gates, overrides, kill switch:
- Audit logs: fields, retention, customer access and export:

## Transparency to end users
- How users know content or actions are AI-generated:
- Content marking (if the system generates media or published text):

## Monitoring and incidents
- What we monitor in production:
- How customers report a problem; how we notify them:

## Change log
| Date | Change | Re-reviewed? |
|---|---|---|
| | | |

# AI red-team plan · [Product] · [System version] · [Dates]

**Owner:** [PM or applied AI lead]   **Security partner:** [name]   **Retest date:** [Date]

## Objective
What decision this exercise informs (for example, beta go or no-go):

## Scope
- System version, environment (never production customer data unless approved):
- In scope: [features, tools, connectors]
- Out of scope:

## Harm and attack categories
| Category | Source (impact assessment, NIST GAI risk, OWASP LLM or Agentic item) | Priority |
|---|---|---|
| | | |

## Red-team types and people
- Expert: [who]
- General users or domain practitioners: [who]
- AI-generated attacks: [approach and volume]

## Scenarios
| # | Scenario | Seed or example | Expected safe behavior | Failure definition |
|---|---|---|---|---|
| 1 | | | | |

## Severity scale
- S1: could cause harm in production or expose customer data
- S2: unsafe or wrong behavior with a plausible path to harm
- S3: policy or quality violation without direct harm
- S4: cosmetic

## Rules of engagement and data handling
-

## Findings
| # | Scenario | Severity | Reproduction | Fix | New eval case | Status |
|---|---|---|---|---|---|---|
| | | | | | | |

## Decision
[Proceed / proceed with conditions / hold], decided by [name] on [date].

# AI incident response plan · [Product] · v[n] · [Date]

Extends the company security incident process. Link: [process]

## AI-specific triggers
- Confident wrong output affecting many users or a high-impact decision
- Unsafe or unintended agent action
- Prompt injection or tool misuse that changed system behavior
- Personal or confidential data exposed through outputs
- Fairness or bias finding
- Unannounced behavior change after a model or provider update

## Severity
| Level | Definition for this product | Example | Response time |
|---|---|---|---|
| 1 | | | |
| 2 | | | |
| 3 | | | |

## Controls available
| Use case | Kill switch or degrade option | Who can trigger | Time to effect |
|---|---|---|---|
| | | | |

## Roles
- Incident lead: [name]
- PM (customer impact, comms, eval follow-up): [name]
- Applied AI (diagnosis, model or prompt rollback): [name]
- Security, privacy, legal: [names]

## Clocks to check (confirm with counsel)
- Contractual notification commitments: [summary]
- GDPR personal data breach notification, if personal data is involved
- EU AI Act Article 73 serious incident reporting, for high-risk systems once applicable
- Other: [jurisdiction or customer specific]

## Customer communication
- First notice within [time], containing: what happened, what we changed, what customers should do, when we will update next.

## After the incident
- [ ] Root cause (data, retrieval, prompt, model, tool, UX, process)
- [ ] New eval cases added and passing
- [ ] System card and impact assessment updated
- [ ] Decision log entry
- [ ] Tabletop scheduled for the next quarter

# AI security questionnaire answer bank · [Product] · [Date]

Structure follows the domains your customers use most (for example, the AI-CAIQ). One canonical answer per question. A named owner approves every answer before it goes to a customer.

| ID | Domain | Question (canonical form) | Answer | Evidence (doc, report, control) | Frameworks satisfied (ISO/IEC 42001, NIST AI RMF, EU AI Act, other) | Owner | Last reviewed | Confidential? |
|---|---|---|---|---|---|---|---|---|
| G-01 | Governance | Do you have an AI policy and a review process? | | | | | | |
| M-01 | Models and suppliers | Which models and providers do you use, and where are they hosted? | | | | | | |
| D-01 | Data use | Do you train on customer data? | | | | | | |
| P-01 | Privacy | Can you support our DPIA? | | | | | | |
| S-01 | Security | How do you defend against prompt injection and tool misuse? | | | | | | |
| E-01 | Evals and testing | How do you test quality, safety and fairness? When was your last red team? | | | | | | |
| H-01 | Oversight | What can the system do without human approval, and how is it stopped? | | | | | | |
| T-01 | Transparency | How do users know they are interacting with AI? | | | | | | |
| I-01 | Incidents | How will you notify us of an AI incident? | | | | | | |
| R-01 | Regulation | What is your EU AI Act role and risk classification? | | | | | | |

## Rules
- Answers cite evidence; if there is no evidence, the answer says what exists and what is planned, with a date.
- New questions not in the bank are answered by hand, then added.
- Review every answer at least every [six] months and after any material change.

# Responsible AI review · [Product] · [Use case] · Gate: [PRD / pre-beta / pre-GA / material change] · [Date]

**Presenter:** [PM]   **Approver (can block):** [name]   **Attendees:** [applied AI, security, privacy, legal, UX]

## Evidence presented
- [ ] Triage with role, tier and transparency duties; counsel's view recorded
- [ ] Impact assessment, current version
- [ ] DPIA status (needed / done / not needed, with reason)
- [ ] Eval results, including slices and regression suite
- [ ] Red-team findings and their status (pre-beta and later)
- [ ] System card and instructions for use (pre-GA and later)
- [ ] Incident plan with kill switches tested (pre-GA and later)
- [ ] Disclosure and content marking in the UX
- [ ] Logging that meets deployer retention needs
- [ ] Questionnaire answer bank updated

## Questions the approver asks
1. What is the worst thing this system can do, and what stops it?
2. Who could be treated worse, and how would we know?
3. What changed since the last review?
4. Which residual risks are we accepting, and who owns them?

## Decision
- [ ] Approved   - [ ] Approved with conditions: [conditions, owners, dates]   - [ ] Not approved: [what is needed]

Recorded in the decision log by [name] on [date].

Prompts

Goal: Get a first-pass risk triage you can take to counsel, with the reasoning visible.
Paste in: a description of the use case (what it does, users, affected people, data, what it can change), your markets, and current source text for the EU AI Act risk tiers and Annex III if you have it. Use an approved tool for unreleased product details.

Act as a product risk analyst. Triage this AI use case for [product].

1. State our likely EU AI Act role (provider, deployer or both) and whether any customer or partner could become a provider through branding, substantial modification or a new high-risk purpose.
2. Screen against: prohibited practices; each Annex III area; profiling of people; the Article 6(3) conditions; Annex I; Article 50 transparency duties; likely DPIA triggers under GDPR.
3. For each screen, give the answer, the reasoning and your confidence. Where the answer depends on facts I haven't given you, say which facts.
4. Propose an internal tier (low, medium, high) and the review track.
5. List the questions I should take to counsel, in priority order.

Rules: present legal screens as questions for counsel. Cite the article or annex supporting each screen. Use dates only from supplied source text and identify missing sources.

[use case description]
[source text]

Check the output: confirm each cited article says what the model claims. Treat the result as a draft for counsel, never as a classification.

Goal: Produce a complete first draft of the impact assessment without starting from a blank page.
Paste in: the PRD and AI PRD addendum, the triage, the eval report, and the impact assessment template. Use an approved tool for internal documents.

Using the template provided, draft an AI impact assessment for [use case].

- Fill purpose, boundaries and prohibited uses from the PRD. Quote the PRD where you take something from it.
- For harms, consider the six principles (fairness, transparency, accountability, privacy, safety, security) and the twelve NIST generative AI risks. Include only harms plausible for this use case, and say why each is plausible.
- For each significant harm, propose a mitigation written as a testable requirement (an eval, gate, log field or UX element).
- Mark every field where the documents give no evidence as "No evidence yet: [what is needed]".

[documents]
[template]

Check the output: every harm should name who is harmed and how. Delete generic harms that would apply to any software. Have the applied AI lead confirm the eval claims.

Goal: Draft the customer-facing system card and catch claims nobody can back up.
Paste in (or connect the harness to): design docs, the connector and tool inventory, permission scopes, the eval report, data retention settings, the DPA, and the system card template.

Draft a system card for [product or feature] using the template. Then produce a second table listing every factual claim in the card, the source document and section it came from, and the engineering owner who must confirm it.

Rules:
- Use only what the sources say. If sources conflict, show both and flag the conflict.
- Where a template field has no source, write "To confirm" rather than filling it in.
- Write for a customer security reviewer: plain, specific, no marketing language.

[sources]

Check the output: send the claims table to the owners and don't publish until every row is confirmed. Pay most attention to data retention, training on customer data and autonomy levels, where a wrong answer becomes a contractual problem.

Goal: Widen red-team coverage beyond what your team would think of.
Paste in: the system card or a description of the agent's tools, permissions, data sources and autonomy levels; the impact assessment's harm list; the OWASP LLM and Agentic Top 10 category names.

You are helping plan an authorized red-team exercise against our own system, [product], in a test environment.

1. For each relevant OWASP LLM and Agentic category, write three realistic attack scenarios specific to this system's tools and data sources. Include at least five where the attack arrives through content the agent reads (logs, tickets, documents, tool outputs).
2. For each scenario, give the expected safe behavior and a precise failure definition.
3. Add five scenarios where nobody is attacking but the system still causes harm (confident wrong output, chained allowed actions, over-reliance).
4. Rank all scenarios by potential severity in our context.

[system description]
[harm list]

Check the output: remove scenarios that don't match the real tools and permissions. Have security review the list and the rules of engagement before anyone runs it.

Goal: Answer a customer AI security questionnaire in days, with every answer traceable.
Paste in: the customer questionnaire, your answer bank, the current system card. This is customer and confidential material: use only your company's approved tool.

For each question in the customer questionnaire:
1. Find the closest entry in the answer bank and adapt the canonical answer to the question's wording. Keep the substance identical.
2. Cite the answer bank ID and the evidence it lists.
3. If no entry fits, write "NEW: needs owner" and suggest which owner should answer, without drafting an answer.
4. Flag any question where the bank's answer is older than its review date.

Output a table: customer question number, draft answer, bank ID, evidence, owner, flags.

[questionnaire]
[answer bank]
[system card]

Check the output: every owner signs their answers before anything is sent. Read every "NEW" item yourself and add the final answer to the bank.

Goal: Find the gaps in your AI incident plan before a real incident does.
Paste in: the AI incident response plan, the system card, and a short description of a past incident or a plausible one.

Run a tabletop exercise for [product] using this scenario: [scenario]. Play the role of facilitator.

1. Reveal the incident in four or five stages, as it would unfold in real life. After each stage, stop and ask me what the team does, who decides and what we tell the customer.
2. After I answer each stage, point out anything the plan doesn't cover: missing severity definitions, kill switches that don't exist, unclear owners, notification clocks we haven't considered.
3. At the end, list the gaps in priority order with a suggested fix for each, and the eval cases the incident should produce.

Keep decisions with me. Ask for my response at each stage, then critique it against the plan.

[plan]
[system card]

Check the output: confirm with counsel any notification deadline the model mentions. Schedule fixes for the top three gaps before the next tabletop.

Goal: Stop out-of-date regulatory facts from reaching customers or roadmaps.
Paste in: the claim (from a deck, a doc or a model's answer) and the current official or consolidated source text you have for it.

Here is a claim about AI regulation: [claim]. Here is the source text I have: [source text].

1. Say whether the source text supports the claim, contradicts it or doesn't address it. Quote the relevant passage (short).
2. If dates are involved, list each date in the claim and the date in the source.
3. Say what you would need to check next if the source doesn't settle it, such as an amending act or a later version.

Rules: use only the source text provided. Don't fill gaps from memory, and say plainly when the source is silent.

Check the output: read the quoted passage in the source yourself. For anything going to a customer, get counsel to confirm.

Anti-patterns

Checklist

• Every AI use case in the product is inventoried separately, with model, provider, data and what it can change.
• Each use case has a triage recording role, Annex III screen, transparency duties and internal tier, and counsel has reviewed anything near a high-risk area.
• Each principle has a product question, an owner and an evidence artifact.
• Use cases above the lowest tier have an impact assessment that cross-references the DPIA and any customer FRIA needs.
• Significant harms are product requirements with acceptance tests in the backlog.
• AI disclosure and, where relevant, machine-readable content marking are in the UX.
• Logs support at least six months of retention by deployers, and agent actions are audited.
• Agents have least-privilege tools, approval gates for hard-to-reverse actions and a tested kill switch.
• A red team covering the relevant OWASP LLM and Agentic categories ran before beta, and its findings became eval cases.
• A customer-facing system card exists, every claim confirmed by its owner.
• An AI incident plan with severities, kill switches and notification clocks exists and has been tabletopped.
• The questionnaire answer bank is mapped to a standard structure, with owners and review dates.
• Review gates are set at PRD, pre-beta, pre-GA and material change, with a named approver who isn’t the PM.
• Regulatory dates and state laws are rechecked quarterly against current sources.