Pro Domain 6 Β· Stakeholder Communication & Lifecycle Management (14%)
This domain is about the work that happens around the code. The architect does six things here. First, find out what the business really needs. Second, make decisions based on evidence. Third, tell each audience the right thing in the right form. Fourth, agree on what "good" means, written as numbers. Fifth, give the building team a design they can build. Sixth, keep checking the system after launch. The MQC profile (the exam's description of the target candidate) calls this "translating business requirements into scalable, secure, and reliable architectures" and "stakeholder engagement, advising clients or internal teams." There is no Foundations domain underneath this one. Foundations taught you how the system works inside β the agent loop, the escalation design, the reliability checklist. This page teaches the documents that decide which of those systems gets built, and when.
The one idea to remember: every consulting document on this page changes something concrete in the Claude architecture. A discovery answer removes a tool. An SLA row chooses a model tier and sets a timeout. An ADR fixes the shape of the workflow. A handoff document names the prompt file and the eval gate. You must be able to point at the field, file, or gate that a document changes. If you cannot, the document is only for show. The exam's wrong options are built from such documents: technically correct documents that go to the wrong audience or are written in the wrong phase.
π New here? One engagement, five documents, one system
Larkspur Insurance wants "an AI that handles claims." The VP of Claims says this in the kickoff meeting. One architect starts drawing an agent with a payout tool that same afternoon. That architect has already failed this domain. Another architect asks twelve questions in a fixed order, writes the answers as numbered requirements, and gets two people to sign them. That architect has started correctly.
Discovery changes the design before the design exists. Question four on the ladder is "where does a person decide, and what can they not hand to someone else?" The answer is "adjusters approve every payout; the finance system pays after a 24-hour hold." That sentence becomes requirement R3: never issue or promise a payout. R3 has three effects. It deletes the issue_payout tool from the draft. It forbids amounts in the letter template. It removes a whole approval-queue design that would otherwise have used up a sprint.
Next, the architect makes the decision with evidence and writes it twice, for two audiences. The engineers get an architecture decision record: the context, three options measured on a golden set, the choice, and the consequences. The CFO gets one page: the same three options as a table with accuracy, latency, and cost compared to the baseline, plus one recommendation. It is the same decision in a different document. The exam tests whether you know which document goes to which reader.
The SLA is agreed as numbers, and the numbers choose the system design. "p95 first token under 3 seconds" is not a wish. It is the reason for four design choices: classification runs on the faster tier, letter drafts stream, the client has an 8-second timeout with a fallback path, and extended thinking is off on the draft step. A stakeholder who wanted "instant" learns what instant costs, and signs a number instead.
The handoff is a build order, not a vision statement. The platform team receives prompt file names with versions, the tool list with read-only scopes, the eval thresholds that each prompt must pass, and the runbook. Then monitoring starts, and monitoring never ends. Every weekly review can start an iteration: a failure cluster becomes a change request, and the change ships behind the same eval gate.
What this means for the exam: questions in this domain describe a moment ("the CFO asksβ¦", "two weeks after launchβ¦", "the team wants to start buildingβ¦"). They ask which document or which action fits that moment. The wrong options are real documents, but at the wrong moment or for the wrong reader.
Which artifact must exist before the next phase may start
Structured discovery and requirement gathering
Conduct structured discovery and requirement gathering.
Who does what. The architect owns the script and runs the sessions. The business owner answers rungs 1, 2, and 6. The ops lead answers rungs 2, 3, and 5. IT answers rung 3 (data shapes). The privacy or security officer answers rung 4. Every answer goes into the requirements sheet, with the person and the date as its source. The sheet lives in the engagement repository next to the ADRs. It does not live in a slide deck: you cannot diff slides, and nobody signs a slide.
Why a ladder. Each rung exists because its answer changes a specific part of the Claude design. Rung 2 (current process, decisions that cannot be handed over) decides which tools are offered and where a human-in-the-loop gate is required. Rung 3 (volume, peak hour, longest document) decides three things: whether the [batch API](term:batch API) is an option, what the [context window](term:context window) must hold, and whether documents arrive as document blocks or as extracted text. Rung 4 decides region, egress, and logging. Rung 5 decides how much [graceful degradation](term:graceful degradation) to build. Rung 6 becomes the success metric and the first row of the SLA table.
YAML
discovery_script:
engagement: Larkspur Insurance β claims intake assistant
rung_1_outcome:
- What decision or action should be faster or better after this ships?
- How is that measured today, and who owns the number?
rung_2_current_process:
- Walk me through one claim from first contact to payout.
- Where does a person decide, and what can they NOT delegate?
- What happens to a claim nobody touches for 48 hours?
rung_3_volume_and_shape:
- How many claims per day; what is the peak hour; what is the longest document?
- Which inputs are structured forms and which are free text or scans?
rung_4_constraints:
- Which data must never leave your region or reach a third party?
- Which regulations and internal policies apply to claim text?
rung_5_failure_tolerance:
- What is the cost of one wrong classification? Of a 10-second delay?
- Who is paged when the assistant is down, and what do they do instead?
rung_6_success:
- What number, at what date, makes this a success to your sponsor?
- What would make you switch it off?
Run the ladder from top to bottom. Rung 6 comes last on purpose. Stakeholders give a realistic success number only after they have described their own process and its failures out loud.
The requirements capture sheet is the document that ends discovery. Each row has an ID, a type, a named source, a priority, and a design consequence. Most teams forget the design consequence column. If a row has no design consequence, the row is either out of scope or not yet understood.
MARKDOWN
# Requirements capture β Larkspur claims intake (v0.3, discovery exit)
| ID | Requirement | Type | Source | Priority | Design consequence |
|---|---|---|---|---|---|
| R1 | Classify each new claim into one of six lines of business | Functional | Ops lead, 2026-09-02 | Must | Classification step; closed enum in the output schema |
| R2 | Draft the acknowledgement letter for adjuster review | Functional | Ops lead, 2026-09-02 | Must | Generation step; template + tone rules in the system prompt |
| R3 | Never issue or promise a payout | Constraint | Claims director, 2026-09-03 | Must | No payout tool; template forbids amounts and dates |
| R4 | p95 response under 3 s in the adjuster console | SLA | Ops lead, 2026-09-03 | Must | Faster tier for R1; streaming for R2; timeout + fallback |
| R5 | Scanned PDFs up to 40 pages | Data | IT, 2026-09-04 | Should | Document block with page cap; batch path for backlog |
| R6 | Claim text and PII stay in-region | Compliance | Privacy officer, 2026-09-04 | Must | Region-pinned endpoint; no third-party logging of prompt text |
| OUT-1 | Fraud scoring | Out of scope | Claims director, 2026-09-03 | β | Existing rules engine owns it; assistant reads its flag only |
Signed: VP Claims (sponsor) Β· Claims ops lead Β· 2026-09-05
R3 removes a tool. Here is what that looks like in the request. The first sketch offered three tools: lookup_claim, draft_letter, and issue_payout with a confirmation step. After discovery, the tool list in the tools field has only one read-only capability. Removing a tool is a stronger control than putting an approval gate in front of it. The model cannot request a capability that is not offered, a person cannot approve it by mistake, and an attacker cannot misuse it through injected text. The exam guide's least-privilege sample uses the same logic (removal beats compensating controls). Here the team reaches it through discovery instead of a security review.
JSON
[
{
"name": "lookup_claim",
"description": "Read-only. Fetch the claim header, line of business if already set, and document list for a claim number. Cannot modify any record.",
"input_schema": {
"type": "object",
"properties": {
"claim_number": {
"type": "string",
"description": "Larkspur claim number, format LK-nnnnn"
}
},
"required": ["claim_number"],
"additionalProperties": false
}
}
]
π Production shape: the one tool as a function
The SDK generates exactly this JSON from a typed function. The Field(description=...) in the Annotated hint becomes the property description. Removing a tool in production means the function is never passed to tools.
PYTHON
import os
from typing import Annotated
import anthropic
from anthropic import beta_tool
from pydantic import Field
from app.claims import claims_repo # your read-only claims repository
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-5")
@beta_tool(
name="lookup_claim",
description="Read-only. Fetch the claim header, line of business if already set, and document list for a claim number. Cannot modify any record.",
)
def lookup_claim(claim_number: Annotated[str, Field(description="Larkspur claim number, format LK-nnnnn")]) -> str:
return claims_repo.header_with_documents(claim_number).to_json()
def classify(claim_number: str):
runner = client.beta.messages.tool_runner(
model=MODEL, max_tokens=600,
tools=[lookup_claim], # draft_letter and issue_payout are not offered
messages=[{"role": "user", "content": f"Classify claim {claim_number}."}],
max_iterations=3,
)
return runner.until_done()
Not supplied in this example β defaults and disabled features
What the SDK generates (from lookup_claim.to_dict()): the block above, plus "title": "Claim Number" on the property.
Where it lives: app/claims/tools.py + app/claims/intake.py.
The letter itself is not a tool. R2 is a generation step: the model writes text, and that text goes to an adjuster's screen. So it needs a template in the system prompt and a review step. It does not need a tool with a side effect.
Discovery finding
Right response
Term
A decision is non-delegable ("adjusters approve every payout")
Remove the capability; if it must exist, gate it before execution
The wrong choice. There are three common failures. First, building a prototype before discovery ("let's build a demo for next week"). The design then follows what the model can do, not what the business allows. The payout tool survives into the pilot, and the first compliance review stops the project. Second, collecting requirements as meeting notes with no sources and no consequences. Three months later nobody can say who said payouts were out of scope, and the team argues about the requirement again. Third, asking one good question (data residency) and calling it discovery. One constraint is one rung, not the whole ladder.
Communicating decisions and trade-offs
Communicate architectural decisions and trade-offs.
Who writes and who reads. The architect writes both documents. The engineers who will live with the consequences review the ADR. The ADR is stored in the repository (docs/adr/), so it is versioned together with the code it explains. The memo goes to the sponsor and to the person who controls the budget. The memo has a date because it expires: the next checkpoint measures its numbers again. Neither document can replace the other. The exam's favourite wrong option is sending one document to the other document's reader.
An ADR is small and permanent. The context lists the requirement IDs that forced the decision. The options are measured, not only described: a [golden set](term:golden set) number, a p95, a cost ratio. The consequences say what becomes harder, because the future reader needs exactly that.
MARKDOWN
# ADR-004 Β· Tiered workflow instead of a single-agent design for claims intake
**Status:** Accepted Β· 2026-09-09 Β· Owners: solution architect, claims ops lead
**Context:** R1βR6. Peak 1,400 claims/hour. p95 < 3 s in the console (R4). Payout is out of scope (R3), so no write tools are needed.
**Options considered**
1. One agent, all capabilities, larger tier β simplest to build. Measured p95 6.8 s on 400 audited claims; violates R4.
2. Fixed workflow: faster-tier classification (structured output, closed enum) β larger-tier letter draft, streamed β p95 2.1 s; 94% classification agreement; two prompts to maintain.
3. Rules engine for classification + model for drafting β cheapest; 71% classification agreement on the same 400 claims.
**Decision:** Option 2.
**Consequences**
- Two prompt files with separate golden sets and separate release gates.
- Routing and fallback code is owned by the platform team, not the prompt owners.
- Revisit when a faster tier reaches β₯ 4.2/5 on the letter rubric (then option 2 collapses to one tier).
**Supersedes:** ADR-002 (single-agent prototype).
The executive memo is one page and asks for one thing. It never explains how prompt caching works. It shows the options as a table that the reader can scan in one minute. It states the recommendation. It also says what is not being decided today β this part prevents the next argument.
MARKDOWN
# Claims intake assistant β architecture recommendation
**To:** CFO, VP Claims Β· **From:** Solution architecture Β· **Decision needed by:** 2026-09-19
**Ask:** approve Option B for the October pilot.
| | A Β· Single agent, larger tier | B Β· Tiered workflow (recommended) | C Β· Rules + model drafting |
|---|---|---|---|
| Classification agreement (400 audited claims) | 95% | 94% | 71% |
| Adjuster-facing response, p95 | 6.8 s β misses the 3 s commitment | 2.1 s | 1.4 s |
| Monthly run cost at current volume | 3.1Γ baseline | 1.0Γ baseline | 0.6Γ baseline |
| Main risk | Slow console; adjusters bypass the tool | Two prompts to maintain | Misrouted claims create rework |
**Recommendation:** B. It meets the 3 s commitment with accuracy within one point of A at a third of A's cost.
**Not being decided today:** payout automation (out of scope per R3); fraud scoring (rules engine, OUT-1).
**Next checkpoint:** pilot review 2026-11-05; the table above is re-measured on live traffic.
"Baseline" means a ratio compared to the current manual process. This is on purpose. Absolute prices go out of date quickly, and they start an argument about pricing instead of a decision about architecture.
Audience asks
Artifact
Why not the other one
"Why did we pick a workflow over an [agent](term:agentic system)?" (engineer, month 6)
ADR
The memo has no rejected options or consequences
"Should we pay for the larger tier?" (CFO, this week)
Trade-off memo
The ADR is too long and asks for nothing
"Which model version are we on and why?" (platform on-call)
ADR + [model pinning](term:model pinning) note in the architecture doc
A chat message is not a record
"Is this safe to launch?" (privacy officer)
Requirements rows R6 + architecture doc data view
The memo is not evidence
"What did we agree the assistant will not do?" (support director)
The wrong choice. A 4,000-word ADR e-mailed to the CFO gets no decision, and the pilot is delayed by a month. A memo with a recommendation but no options gets reversed the first time someone asks "did you consider�" A decision made in a meeting and never written down exists as "[decision as data](term:decision as data)" only in people's memories. When the architect leaves, the new engineer runs the whole evaluation again to answer a question that the ADR would have answered in one paragraph.
Feedback loops, expectation alignment, and SLAs
Manage stakeholder feedback loops and expectation alignment (including SLAs).
Who negotiates. The architect proposes the table. The ops lead owns the measurement. The sponsor signs. The privacy officer owns the residency row. The platform team owns the "measured where" column, because a number that nobody measures is not an SLA. The table lives in the architecture document. Each month it is issued again as the SLA report.
Every SLA row has a design lever. This is the test of a good row: when a number changes, something in the request, the client, or the release gate changes with it.
MARKDOWN
| Indicator (SLI) | Objective (SLO) | Contractual (SLA) | Measured where | Design lever |
|---|---|---|---|---|
| Time to first token, adjuster chat | p95 β€ 2.0 s | p95 β€ 3.0 s, monthly | Gateway log per request | Faster tier for classification; streaming on drafts |
| End-to-end letter draft | p95 β€ 6 s | p95 β€ 8 s, monthly | Console telemetry | max_tokens sized to the template; no extended thinking on drafts |
| Availability (successful responses) | 99.7% | 99.5%, monthly | Gateway 5xx + client timeouts | 8 s timeout, 2 retries with jitter, fallback to faster tier, then queue |
| Classification agreement with adjusters | β₯ 93% | β₯ 90%, quarterly audit | Weekly 200-claim sample | Golden-set regression gate before each prompt release |
| Data residency | 100% in-region | 100% | Quarterly egress audit | Region-pinned endpoint; no third-party logging |
Row 2 becomes a request. The acknowledgement template is about 350 words, so max_tokens is 600. That is enough for the letter. It is also low enough that a draft that runs too long cannot use up the latency budget. Row 1 is the reason stream is on: the adjuster sees the first line before the letter is finished. Nothing in this request body turns on extended thinking, which is row 2's second lever. This is an illustrative request. The model placeholder stands for the larger tier chosen in ADR-004.
JSON
{
"model": "YOUR_SUPPORTED_MODEL",
"max_tokens": 600,
"stream": true,
"system": "You draft claim acknowledgement letters for Larkspur Insurance adjusters to review. Use the supplied template exactly. Never state, estimate, or imply a payout amount or a payout date. If the claim line is missing, say so in one sentence and stop.",
"messages": [
{
"role": "user",
"content": "Claim LK-20931. Line of business: auto collision. Claimant: R. Alvarez. Documents received: police report, two photos. Draft the acknowledgement letter."
}
]
}
Not supplied in this example β defaults and disabled features
Row 3 becomes client configuration and a fallback path. The timeout and the retry count live in the application, not in the request body. This is illustrative Python. The SDK client accepts a timeout and a retry count for each client. The routing function is the application's own harness code.
PYTHON
from anthropic import Anthropic
client = Anthropic(timeout=8.0, max_retries=2)
FAST_INTENTS = {"classify", "missing_documents"}
def choose_tier(step: str, sla_first_token_ms: int) -> str:
"""Return the tier name the gateway maps to a concrete model id."""
if step in FAST_INTENTS or sla_first_token_ms < 3000:
return "faster"
return "larger"
def draft_with_fallback(step: str, request: dict) -> dict:
"""Try the tier the SLA row selects; degrade rather than fail the adjuster."""
tier = choose_tier(step, sla_first_token_ms=3000)
try:
return call_gateway(tier, request)
except TimeoutError:
if tier == "larger":
return call_gateway("faster", request)
enqueue_for_retry(request)
return {"status": "queued", "message": "Draft will appear shortly."}
π Production shape: real SDK errors, streaming, fallback
Two details change in production. The SDK does not raise Python's TimeoutError; after its own retries it raises anthropic.APITimeoutError (or APIConnectionError, InternalServerError). And row 1 asks for streaming, so the call uses messages.stream. Model ids come from configuration, one per tier.
PYTHON
import os
import anthropic
from app.queue import enqueue_for_retry # your own retry queue
client = anthropic.Anthropic(timeout=8.0, max_retries=2) # reads ANTHROPIC_API_KEY
MODELS = {
"larger": os.environ.get("CLAUDE_MODEL", "claude-sonnet-5"),
"faster": os.environ["CLAUDE_MODEL_FASTER"],
}
RETRYABLE = (anthropic.APITimeoutError, anthropic.APIConnectionError, anthropic.InternalServerError)
def stream_draft(tier: str, request: dict, on_text) -> anthropic.types.Message:
# request holds max_tokens, system and messages; the model comes from the tier
with client.messages.stream(model=MODELS[tier], **request) as stream:
for text in stream.text_stream:
on_text(text) # the adjuster sees the first line early
return stream.get_final_message()
def draft_with_fallback(step: str, request: dict, on_text) -> dict:
tier = choose_tier(step, sla_first_token_ms=3000)
try:
message = stream_draft(tier, request, on_text)
except RETRYABLE:
if tier != "larger":
enqueue_for_retry(request)
return {"status": "queued", "message": "Draft will appear shortly."}
message = stream_draft("faster", request, on_text)
if message.stop_reason in ("max_tokens", "refusal"):
return {"status": "needs_adjuster", "reason": message.stop_reason}
return {"status": "ok", "letter": "".join(b.text for b in message.content if b.type == "text")}
Not supplied in this example β defaults and disabled features
This fragment or unpacked request variable does not prove a field is omitted. Inspect the complete enclosing request; do not infer defaults from missing lines.
Where it lives: app/llm/client.py (client + tiers) + app/claims/drafting.py (fallback).
The fallback to the faster tier is [graceful degradation](term:graceful degradation): a good-enough letter now is better than a perfect letter after the adjuster has given up. The queue is the last step. Row 3 counts a timeout as a failure, but a queued draft that arrives later is not lost work. Retries include jitter (a small random delay), so 1,400 claims an hour do not all retry at the same moment.
Expectation alignment is a written list of what the system will and will not do. The architect agrees it with the business owner before launch, and end users see it. The reason: after launch, the loudest feedback comes from people who expected something the system was never designed to do.
MARKDOWN
## Claims intake assistant β what it will and will not do (agreed 2026-09-12)
**Will:** classify each new claim into one of six lines; draft the acknowledgement letter for adjuster review; flag missing documents against the checklist for that line.
**Will not:** decide coverage; state or estimate a payout; contact claimants; touch claims the fraud engine has flagged.
**Accuracy:** 90% classification agreement with adjusters, measured on a weekly 200-claim audit. Not 100%.
**When it is wrong:** the adjuster corrects the line in the console with one click; corrections feed the weekly review.
**When it is down:** the console shows the manual intake form. Nothing is lost; queued drafts appear when service returns.
**Who owns the number:** claims ops lead reports it monthly; architecture explains any variance and proposes the fix.
The feedback loop is a fixed schedule, not an inbox. Each level of the loop has named people, a fixed input, and the one action it is allowed to take. The daily check watches the SLOs. The weekly review ranks failure clusters and starts iterations. The monthly review signs the SLA report. The quarterly review confirms compliance again and removes capabilities nobody uses.
YAML
feedback_loop:
daily:
who: claims ops lead + on-call engineer
input: dashboard β task success, p95 first token, timeout rate, fallback rate
action: open an issue when any SLO is red two days running
weekly:
who: claims director, two adjuster representatives, architect
input: 200-claim audited sample; adjuster corrections tagged by cause
action: rank failure clusters; agree which one enters the next iteration
monthly:
who: VP Claims (sponsor), CFO delegate, architect
input: SLA report against the contract table; run cost versus baseline
action: confirm, renegotiate a row, or pause; sponsor signs the report
quarterly:
who: privacy officer, security, architect
input: egress audit; access review of every tool scope
action: re-attest the residency row; remove any tool unused for 90 days
Show the metric and the cluster at the weekly review; open an iteration
Roll back or swap tiers on the spot
"Why did it not pay the claimant?"
Expectation sheet not read
Re-issue the will/will-not list; add it to the console
Add a payout tool
"Can we have 99.99%?"
A row with no fallback design behind it
Price the row: multi-region, queueing, on-call; renegotiate
Sign it
"Just switch it off" after one bad week
No agreed cadence for decisions
Point to the monthly action: confirm / renegotiate / pause
Let the loudest voice decide
The wrong choice. If you sign an SLA with no "measured where" column, people argue about the number every month, and nobody ever designs for it. If you treat each complaint as a change request, the prompt changes constantly with no regression gate. Then the fix for the first complaint causes the second complaint. If you skip the expectation sheet, you get the most expensive feedback of all: a support director who "assumed it would handle payouts", and a sponsor who now doubts the whole programme.
Architecture documentation and implementation guidance
Document architectures and provide implementation guidance.
Who writes, who reads. The architect writes both. Builders, reviewers, auditors, and the next architect read the architecture document. It is versioned in the repository together with the ADRs it links to. Implementation guidance is written for one team, and that team's lead accepts it. This acceptance is the exit criterion for handoff. The runbook is written together with the on-call engineer. Other documents link to it; they do not copy it.
The architecture document has a fixed structure. So a reader knows where the SLA table is without searching, and an auditor can go straight to the data view.
MARKDOWN
# Claims intake assistant β architecture (v1.2, 2026-09-16)
1. Purpose and scope β requirements R1βR6; out-of-scope list (OUT-1 fraud scoring)
2. Context β adjuster console β gateway β workflow service β Claude API; claims DB (read-only via lookup_claim); document store
3. Runtime view β step 1 classify (faster tier, structured output, closed enum of six lines);
step 2 draft (larger tier, streamed, max_tokens 600); step 3 adjuster review and one-click correction
4. Data view β what enters each prompt; what is logged (metadata and metrics only, never claim text to third parties);
retention 90 days; region-pinned endpoint (R6)
5. Decision log β ADR-001 model tier selection Β· ADR-002 single-agent prototype (superseded) Β· ADR-003 structured output for classification Β·
ADR-004 tiered workflow Β· ADR-005 timeout and fallback policy Β· ADR-006 golden set ownership
6. Quality attributes β SLA table (Β§ Feedback loops); eval gates per prompt; model pinning and change policy
7. Operations β runbook `ops/runbook.md`; on-call rota; dashboard panels for the five SLIs
8. Open questions and planned revisions β marine line of business (see CR-017)
Implementation guidance is a build order with rules that cannot change. It does three things. It names files and versions, so "the prompt" always means one exact file. It repeats the constraints from the requirements and the ADRs, so the building team does not have to discover them again. And it defines "done" for each step; the eval gates live there.
MARKDOWN
# Implementation guidance β claims intake assistant β Platform team (handoff 2026-09-18)
## Build order
1. Gateway route with the timeout / retry / fallback policy from ADR-005 (owner: platform)
2. Classification step β prompt `prompts/classify.v3.md`, output schema `schemas/claim_class.json`, tool list `tools/readonly.json`
3. Draft step β prompt `prompts/letter.v2.md`, template `templates/ack_letter.md`, streaming on
4. Console integration β the one-click correction must write to `feedback.corrections` with prompt version and model tier
## Non-negotiables (from requirements and ADRs)
- No tool may write to the claims system (R3, ADR-004). The tool list has one read-only entry.
- Region-pinned endpoint only (R6). Prompt and response text never reach third-party logging.
- Both prompts ship behind the golden-set gate: classification β₯ 93%; letter rubric β₯ 4.2 / 5.
- Model tier changes require an ADR; prompt wording changes require a passing eval run, not an ADR.
## Definition of done, per step
- Unit tests for schema validation, timeout, and both fallback branches
- Eval run recorded with prompt version, model tier, golden-set version, and score in `evals/runs/`
- Dashboard panels for the five SLIs live before the first production claim
## Contacts and escalation
- Architecture reviews any change to prompts, tools, tiers, or data handling
- Claims ops lead owns the golden set and the weekly 200-claim audit
- On-call: see `ops/runbook.md` Β§ "Assistant unavailable β manual intake form"
Document
Answers
Owner
Lives in
Architecture document
What the system is, why (decision log), how well (SLA), how run (ops)
Architect
docs/architecture.md
ADR
Why one structural choice was made and what it costs
Architect
docs/adr/
Implementation guidance
What to build first, what may not change, how done is proven
Architect β team lead
docs/handoff/
Runbook
What on-call does when an SLI goes red
Architect with on-call
ops/runbook.md
Eval run log
Which prompt version at which tier scored what on which [golden set](term:golden set)
Platform team
evals/runs/ β the [audit trail](term:audit trail) for every release
Prompt and schema files
The exact text and [closed schema](term:closed schema) the API receives
Prompt owners, gated by evals
prompts/, schemas/
The wrong choice. A handoff that is only a vision paragraph ("an intelligent assistant that streamlines claims") ships a payout tool, because nobody told the team that the tool was removed. A handoff that copies the discovery notes hides the four fixed rules inside forty pages of interviews. An architecture document with no decision log means every future change starts the old argument again. And guidance with no definition of done lets "the classifier works" mean whatever the demo showed.
Lifecycle phases β discovery to iteration
Support lifecycle phases (discovery, design, handoff, monitoring, iteration).
Who controls each gate. The sponsor and the ops lead sign to close discovery. To close design, engineering review accepts the ADRs and the sponsor approves the memo. To close handoff, the building team's lead accepts the guidance. The weekly review starts iterations, and the monthly review can pause the system. The architect attends every gate but never signs one alone. This is on purpose: a gate that one person can pass alone is not a real gate.
Phase
Entry criteria
Main artifacts
Exit criteria
Wrong-phase trap
Discovery
A sponsor and a named business owner
Discovery script; requirements sheet
Sheet signed; success metric and SLA targets written as numbers; out-of-scope list agreed
Prototyping to "find requirements"
Design
Signed requirements
ADRs; trade-off memo; SLA table; expectation sheet; eval plan with a [golden set](term:golden set)
ADRs accepted; memo approved; golden set β₯ 200 items exists
Guidance accepted by the building team; runbook and dashboards live before first traffic
Sending the memo as the handoff
Monitoring
First production traffic
Daily dashboard; weekly audit; monthly SLA report
Never β continuous
Treating a met SLA as "no feedback needed"
Iteration
A ranked failure cluster or SLO breach agreed at the weekly review
Change request; eval run; ADR only if structure changes
Shipped behind the regression gate; re-measured one full cycle
Hot-fixing the prompt from a complaint
The gates as a checklist that the programme lead can run. This is the same table, written as data. A phase may not start until every item in the previous phase's exit_when list is true.
YAML
phase_gates:
discovery:
exit_when:
- requirements sheet signed by sponsor and ops lead
- success metric and SLA targets written as numbers, not adjectives
- out-of-scope list agreed and dated
design:
exit_when:
- ADRs accepted for model tiering, workflow shape, and data handling
- trade-off memo approved by the sponsor
- eval plan with a golden set of at least 200 items, owner named
handoff:
exit_when:
- implementation guidance accepted by the building team's lead
- runbook and dashboards exist before the first production claim
monitoring:
exit_when: never
feeds: iteration
iteration:
entry_when:
- a ranked failure cluster or an SLO breach is agreed at the weekly review
exit_when:
- change shipped behind the eval gate
- re-measured for one full weekly cycle with no new cluster
Iteration is a change request, not a quick fix in production. The weekly review found marine claims that were wrongly sent to auto. The change touches the enum, the golden set, and the prompt. None of these changes the tier, the tools, or data handling, so no ADR is needed. The eval gate is still needed. It includes a separate threshold for the marine subset, so the fix is proven on the exact cluster that caused the change.
YAML
change_request: CR-017
trigger: weekly review 2026-11-12 β cluster "boat/marine claims misrouted to auto" (31 of 200 audited)
phase: iteration
proposed_change:
- add "marine" to the closed enum in schemas/claim_class.json
- add 12 marine claims to the golden set (owner: claims ops lead)
- prompt classify.v3 -> classify.v4 with one marine few-shot example
adr_required: no (no change to tier, tools, or data handling)
eval_gate: classification >= 93% overall AND >= 85% on the marine subset
rollout: shadow for 3 days, compare corrections, then 100%
owner: platform team
reviewer: architecture
re_measure: weekly audit 2026-11-26
The few-shot example and the enum change are Domain 2 and Domain 3 work. The lifecycle decided when and how they happen: now, behind a gate, with a named owner β not on the same day as the complaint.
The wrong choice. Suppose the team starts to build retrieval "because the design is reviewed", but the sheet has no SLA. That is a design-phase decision made without the input that discovery should give. The chosen tier will be wrong, and the ADR will be rewritten. If the team treats monitoring as a phase that ends at launch, the sponsor finds the first failure cluster. If the team skips the change request and edits the prompt directly, a met SLA becomes a missed one: there is no gate, no new measurement, and no record of what changed.
See it run β one engagement end to end
π See it run β Larkspur claims intake, from the first question to the first change request
Discovery, day 2 β rung 2 answer. Claims director: "Adjusters approve every payout. Finance pays after a 24-hour hold. The assistant should sort and draft, nothing else." The architect writes the row on the spot:
MARKDOWN
| R3 | Never issue or promise a payout | Constraint | Claims director, 2026-09-03 | Must | No payout tool; template forbids amounts and dates |
Discovery, day 3 β the tool list changes. The sketch had three tools; now it has one. The architect deletes the issue_payout entry from tools/readonly.json. The approval-queue design that would have gated that tool is removed from the plan. Nothing has been built yet. That is exactly why this work belongs in discovery.
Discovery exit, day 5. The sheet has seven rows, a success number (rung 6: "90% classification agreement by the end of the pilot"), and two signatures. Design can start.
Design, week 2 β three options are measured. The golden set has 400 audited claims. The team runs each option against it. p95 is measured in the console, not in a notebook. The numbers go into ADR-004 for the engineers and into the memo for the CFO.
Design, week 3 β the SLA row chooses the design. Row 1 (p95 first token β€ 3 s) puts classification on the faster tier. Row 2 sets the size of the draft request:
JSON
{
"model": "YOUR_SUPPORTED_MODEL",
"max_tokens": 600,
"stream": true,
"system": "You draft claim acknowledgement letters for Larkspur Insurance adjusters to review. Use the supplied template exactly. Never state, estimate, or imply a payout amount or a payout date.",
"messages": [
{
"role": "user",
"content": "Claim LK-20931. Line of business: auto collision. Claimant: R. Alvarez. Draft the acknowledgement letter."
}
]
}
Not supplied in this example β defaults and disabled features
Row 3 (99.5% availability) becomes Anthropic(timeout=8.0, max_retries=2) plus the fallback function. ADR-005 records this policy. So the on-call engineer knows why a slow draft sometimes comes from the faster tier.
Design exit, week 3. The sponsor approves option B on the memo. Engineering accepts ADR-001 to ADR-006. The support director agrees to the expectation sheet. Handoff can start.
Handoff, week 4. The platform lead accepts the implementation guidance. The dashboard has five panels before the first production claim. The team tests the runbook's "assistant unavailable" section: they switch the gateway off for ten minutes and check that the manual form appears.
Monitoring, week 9 β the alert that is not an emergency.
The daily check opens an issue. Nobody rolls anything back, because the SLA row is still met, and the schedule says the weekly review makes the decision.
Weekly review, week 10. The 200-claim audit shows 31 marine claims classified as auto. Marine is a line of business that the requirements never listed. The review ranks this cluster first and opens CR-017.
Iteration, weeks 10β12. The enum gets "marine". The golden set gets 12 marine claims. The prompt gets one example. The eval gate passes at 94% overall and 88% on the marine subset. The change runs in shadow mode for three days, then ships. The next audit measures it again: 95%. The monthly SLA report records the dip, the cause, and the fix. Monitoring continues.
What changed in the Claude architecture at each step: a tool was removed (discovery); each step got a tier, and the draft became a streamed request with a sized max_tokens (design); the client got a timeout and a fallback (design); the enum and a few-shot example changed (iteration). You can trace each change back to a requirement row, an ADR, or a change request.
Exam lens β how CCAR-P asks about this domain
Questions here describe a moment and ask for the document or action that fits it. The wrong options are real documents, but at the wrong moment or for the wrong reader. Before you read the options, find the audience and the phase.
"In the kickoff meeting the VP asks forβ¦" / "before proposing an architecture" β discovery objective. The right answer runs the structured ladder and writes requirements. The wrong options build a prototype, choose a tier, or ask one good question and stop.
"During discovery the team learns that X is never done by the assistant" β a requirement that removes a capability. Removal is better than a gate (HITL), and a gate is better than logging. The guide's words for this reasoning: "removal of unnecessary privilege" versus "detective/compensating controls."
"The CFO / sponsor / executive asks whetherβ¦" β a one-page trade-off memo with three options and a recommendation. An ADR, an eval notebook, or a chat summary are wrong options because they go to the wrong audience.
"A new engineer asks why the team choseβ¦" / "the original architect has left" β the ADR. The words that point to it are context, options considered, consequences.
"Two weeks after launch the director says it gets things wrong", and the metric is above the agreed target β feedback loop and expectation alignment. Show the agreed metric and the failure cluster at the scheduled review, then start an iteration with a clear, limited scope. Rollback, a tier swap, and escalating over the director's head are wrong options because they ignore the agreed schedule.
"The SLA requires p95 β¦ / 99.5% availability", and the current design uses the largest tier with thinking on for everything β design levers driven by the SLA: a tier for each step, streaming for first token, timeouts with fallback, max_tokens sized to the output. Raising max_tokens or adding few-shot examples does not change a latency or availability number.
"The team wants to start building; the requirements sheet has no success metric" β lifecycle exit criteria. Go back to discovery and agree the number. Do not build a prototype to find it.
"Which content belongs in the handoff?" β files with versions, tool scopes, eval thresholds, runbook, contacts. Vision text, interview notes, and the executive memo are wrong options because they are the wrong document.
Reasoning words the answer key uses:wrong audience (right document, wrong reader), wrong phase (right action, wrong moment), removal versus compensating control, agreed metric versus a single story, fixed schedule versus reaction, buildable versus descriptive. When two options both look correct, the one that names a number, a file, or a signed document usually beats the one that only names an intention.