🎓 Claude Architect KB CCAR-F · CCAR-P study guide

Concept Drills — 120 Questions That Test the Concept, Not the Shape

Exam items rarely reuse a scenario you have seen; they reuse the concept behind it. Each drill here tests one concept from the exam blueprint through a fresh scenario, a different angle and its own options, so recognising a question's shape stops being enough. Name the concept first, then choose. Where a question raised a doubt during review, the resolved clarification sits beneath it.

Jump to a domain: Agentic Architecture · Claude Code · Prompting & Structured Output · Tools & MCP · Context & Reliability

These drills share one grading file with the other practice pages; resetting a domain here never touches another page's score.

Drill · Agentic Architecture (Q220–Q249)

Q220. A freight company's coordinator asks a subagent to normalise 40,000 shipping addresses. The delegation prompt is eleven numbered steps (split on commas, uppercase the third field, pad postcodes to five digits, and so on). Partway through, the subagent meets a new region whose addresses contain no commas, keeps executing step two anyway, and reports success on thousands of mangled rows. Which diagnosis is exactly right? A) The subagent had too many tools available, which distracted it from the address-parsing steps it was supposed to follow in order. B) The subagent ran out of output budget while reasoning about the new format, so it fell back to the literal steps it had been given. C) The steps were sound; the prompt only needed a rule telling the subagent to halt and ask the coordinator whenever input deviates from the script. D) The prompt specified how to work but never what a correct result looks like, so when the data broke the script the subagent had no criterion to notice failure or adapt toward.

Understand Q220 · the situation and each part of the question

The real situation behind this question

A freight company is cleaning its address book before switching carriers, and a bad address means a failed delivery and a support call. The coordinator wanted control, so it scripted every move for the worker. When addresses from a newly acquired region arrived without commas, the script silently produced garbage, and the worker had no idea anything was wrong because nobody had told it what a correct address looks like.

Read the question in parts

Part 1 · eleven numbered steps

The prompt is procedural: it fixes the method and never defines the result the worker is responsible for.

Part 2 · reports success on thousands of mangled rows

The worker could not detect failure, which points to missing success criteria rather than a lack of effort or tools.

What you are being asked to decide

Why did a step-by-step delegation prompt fail when the data changed? Tests writing subagent prompts as goals plus success criteria rather than procedures.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Subagentthe address worker that received eleven steps but no picture of a good result
Handoffthe delegation prompt; it should carry the goal and the checks for done
Orchestratorthe coordinator that wrote the step list instead of the outcome

Study: Coordinator communication from plan to final report Subagent Handoff Orchestrator

Understand Q220's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The worker obeyed the script and still failed, because a script cannot anticipate every input and it gave no definition of a correct result. Delegate the outcome plus checks the worker can verify, and let it choose the method.

Connect the design to the situation

A subagent follows address-cleaning steps literally on data the steps never anticipated.

Best answer: D. The subagent did exactly what the steps said, which is why it failed: nothing in the prompt described a correct address, so it could not see the damage or choose a different method. A halt-and-ask rule keeps the brittle script and still leaves success undefined.

What each choice would do here

A · Why this does not meet the full requirement

The subagent had too many tools available, which distracted it from the address-parsing steps it was supposed to follow in order.

Tool scoping matters, but the failure here is the missing definition of done.

Trimming tools is a valid hygiene step for tool-selection errors. Here the subagent used the right tool exactly as instructed; the instruction itself was the problem, so a smaller menu would change nothing.

B · Why this does not meet the full requirement

The subagent ran out of output budget while reasoning about the new format, so it fell back to the literal steps it had been given.

Budget did not cause this; the subagent never had a reason to question step two.

Raising max_tokens gives longer replies but changes nothing about what the subagent is aiming for. It would still execute the numbered steps faithfully and still report success on rows that the script mangled.

C · Why this does not meet the full requirement

The steps were sound; the prompt only needed a rule telling the subagent to halt and ask the coordinator whenever input deviates from the script.

A halt rule keeps the brittle script and still gives no definition of a good result.

An escalate-on-deviation rule turns every regional quirk into a round trip to the coordinator. The subagent still cannot tell good output from bad, so any deviation the script did not anticipate slips through silently.

D · Why this meets the requirement

The prompt specified how to work but never what a correct result looks like, so when the data broke the script the subagent had no criterion to notice failure or adapt toward.

Right: a procedure without success criteria cannot detect its own failure or guide adaptation.

Rewriting the handoff as a goal plus checks (every record parses into street, city, postcode; unparseable rows are returned, not guessed) lets the subagent change method for the comma-less region and still know when it is done.

Check your understanding · explain the design in your own words

Why did a step-by-step delegation prompt fail when the data changed?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Coordinator communication from plan to final report

Study: Coordinator communication from plan to final report Subagent Handoff Orchestrator

Return to Q220's situation and clause-by-clause explanation

Q221. A documentation platform's coordinator receives two jobs at once: correct a misspelled word in one README heading, and audit 400 API reference pages for broken cross-links. The coordinator already has the README open. Which assignment keeps coordination cost lowest while keeping the coordinator's own context clean? A) Two subagents, one per job, so the coordinator stays a pure router and never edits files itself. B) A subagent fixes the heading, while the coordinator audits the pages because link checking needs the global view only it holds. C) The coordinator fixes the heading itself and hands the 400-page audit to one subagent that returns only the list of broken links. D) The coordinator does both, reading all 400 pages into its own context so it has a complete picture for future requests.

Understand Q221 · the situation and each part of the question

The real situation behind this question

A docs team runs a coordinator agent that fields maintenance requests. One morning two requests land together: a typo in a README heading and a full link audit ahead of a major release. The team pays for tokens and waits on latency, and the coordinator must stay sharp for the rest of the day's requests. Deciding what to delegate is a cost judgement, not a matter of keeping roles pure.

Read the question in parts

Part 1 · correct a misspelled word in one README heading

The first job is tiny and single-location, so a handoff would cost more than the work itself.

Part 2 · keeps coordination cost lowest while keeping the coordinator's own context clean

Two constraints at once: minimise handoffs and avoid flooding the coordinator, which points to delegating only the bulky job.

What you are being asked to decide

Which of two jobs should the coordinator delegate, and which should it just do? Tests weighing handoff cost against the benefit of isolated context.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Context isolationkeeping 400 pages of link checks out of the coordinator's working context
Handoffthe brief-and-collect cost paid every time work goes to a subagent

Study: Choosing the simplest sufficient design Subagent Context isolation Handoff

Understand Q221's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Delegation costs a brief, a spawn and a result collection, and repays that only when the work is large or noisy enough to deserve its own context. Do the one-word fix inline and send the 400-page audit to a subagent that returns just the broken links.

Connect the design to the situation

A one-word fix and a 400-page link audit arrive together.

Best answer: C. A subagent buys a fresh, focused context at the price of a handoff, so it pays off for bulky work like a 400-page audit and not for a one-word fix. Sending both jobs out pays the handoff twice for no gain.

What each choice would do here

A · Why this does not meet the full requirement

Two subagents, one per job, so the coordinator stays a pure router and never edits files itself.

Delegating a one-word fix pays full handoff overhead for almost no work.

Spawning a worker for the typo means writing a brief, starting a context, and collecting a result for an edit the coordinator could make in one step. Purity of role is not worth that overhead.

B · Why this does not meet the full requirement

A subagent fixes the heading, while the coordinator audits the pages because link checking needs the global view only it holds.

This inverts the cost logic: the large job needs isolation, the small one does not.

Link checking needs the page set and link targets, which a subagent can be given. Keeping the heavy job in the coordinator while outsourcing the trivial one maximises both context load and handoff cost.

C · Why this meets the requirement

The coordinator fixes the heading itself and hands the 400-page audit to one subagent that returns only the list of broken links.

Right: the tiny edit is cheaper inline; the large audit benefits from its own context and returns a compact result.

The one-word fix costs a single edit where the file is already open. The audit would flood the coordinator with 400 pages, so isolating it in a subagent keeps that noise out and returns only what matters.

D · Why this does not meet the full requirement

The coordinator does both, reading all 400 pages into its own context so it has a complete picture for future requests.

Inlining the audit bloats the coordinator's context with pages it will never need again.

Four hundred pages of link checking would crowd out the coordinator's planning context and degrade later decisions. The audit is exactly the kind of bulky, self-contained work that deserves isolation.

Check your understanding · explain the design in your own words

Which of two jobs should the coordinator delegate, and which should it just do?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choosing the simplest sufficient design

Study: Choosing the simplest sufficient design Subagent Context isolation Handoff

Return to Q221's situation and clause-by-clause explanation

Q222. A market-intelligence system is asked to profile the payment methods used by consumers across Southeast Asia. The final report covers credit cards and bank transfers in depth but never mentions e-wallets, buy-now-pay-later or cash on delivery. Reviewers agree each worker's section is thorough and well sourced. Which single piece of evidence would most directly confirm where the gap originated? A) The synthesis worker's prompt, to check whether it was told to flag categories missing from its inputs. B) The retrieval index, to verify whether documents about e-wallets and buy-now-pay-later exist in the corpus at all. C) The coordinator's delegation record showing the list of subtopics it assigned before any worker started. D) The search worker's query log, to see whether its queries were phrased too narrowly to surface wallet providers.

Understand Q222 · the situation and each part of the question

The real situation behind this question

A fintech is deciding which checkout options to build for its regional launch, and the market-intel report is the input to that roadmap. In much of Southeast Asia, e-wallets and cash on delivery dominate. A report that silently omits them could send the product team to build the wrong features. The team wants proof of where the omission began before changing anything.

Read the question in parts

Part 1 · each worker's section is thorough and well sourced

Workers performed well within scope, which rules out worker quality as the origin of the gap.

Part 2 · where the gap originated

The question asks for the earliest point of failure, not a control that would catch the gap later.

What you are being asked to decide

Where would you look to prove why the report missed whole payment categories? Tests tracing a coverage gap to the coordinator's task decomposition.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Decompositionthe coordinator's split of 'payment methods' into subtopics, which left three out
Hub-and-spokeworkers receive scope only from the coordinator, so its plan bounds what they can find

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure Decomposition Orchestrator Hub-and-spoke

Understand Q222's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When every worker did good work and whole categories are still missing, the categories were never assigned. Read the coordinator's delegation record first: it is the earliest point where the gap could appear, and fixing coverage planning there fixes everything downstream.

Connect the design to the situation

A payments report covers cards and transfers but skips wallets, BNPL and cash on delivery.

Best answer: C. Thorough workers plus missing whole categories means nobody was assigned those categories, and the coordinator's delegation record is the evidence that shows that directly. Query logs and synthesis prompts describe behaviour inside the assigned scope, not how the scope was chosen.

What each choice would do here

A · Why this does not meet the full requirement

The synthesis worker's prompt, to check whether it was told to flag categories missing from its inputs.

A gap-flagging synthesis is a useful detective control, but it is downstream of the origin.

Even a perfect gap-flagging prompt would only report the hole after the fact. The question asks where the gap originated, and synthesis can only summarise what the workers were assigned and returned.

B · Why this does not meet the full requirement

The retrieval index, to verify whether documents about e-wallets and buy-now-pay-later exist in the corpus at all.

Source availability is irrelevant if no worker was ever asked to look for those methods.

The corpus almost certainly contains wallet documents. Confirming that would show the sources were available, but it still leaves open why no worker went looking, which is the planning question.

C · Why this meets the requirement

The coordinator's delegation record showing the list of subtopics it assigned before any worker started.

Right: if the missing methods were never assigned, the decomposition is the origin.

Workers only research what they are handed. If the delegation record lists only cards and transfers, the gap was created at planning time and no downstream component could have filled it, which confirms the root cause directly.

D · Why this does not meet the full requirement

The search worker's query log, to see whether its queries were phrased too narrowly to surface wallet providers.

Query scope matters only inside an assignment; it cannot explain topics nobody assigned.

The query log would show thorough searches for cards and transfers, because that is what the worker was asked about. It tells you about search quality within scope, not about why three whole payment categories never entered scope.

Check your understanding · explain the design in your own words

Where would you look to prove why the report missed whole payment categories?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure Decomposition Orchestrator Hub-and-spoke

Return to Q222's situation and clause-by-clause explanation

Q223. A support agent's loop ends when the assistant's text contains the sentinel "FINAL ANSWER:". Some tickets end mid-task because the agent quotes a customer email that happens to contain that phrase; others grind on until the 25-turn safety cap even though the work finished earlier. Which statement about the loop's exit condition is exactly right? A) A rarer sentinel such as a random token would fix both failures, because customers could never type it by accident. B) Every response already carries a stop_reason that separates wanting a tool (tool_use) from being finished (end_turn), so the loop should branch on that field rather than on text. C) Keep the sentinel but normalise case and require it at the very end of the message, so quoted text cannot trigger it. D) The 25-turn cap is the real defect; raising it lets long tickets reach the sentinel before the loop gives up.

Understand Q223 · the situation and each part of the question

The real situation behind this question

A help-desk team built its first agent loop quickly and used a text sentinel to decide when to stop. In production, customers paste email threads into tickets, and some of those threads contain the same phrase. Other tickets finish early but keep calling the model until the cap. The team is paying for wasted turns and closing tickets with half-done work, and wants to know what the exit condition should really be.

Read the question in parts

Part 1 · ends when the assistant's text contains the sentinel

Completion is being inferred from prose, which the model or the user can produce for unrelated reasons.

Part 2 · grind on until the 25-turn safety cap

The loop misses real completion, showing the text check is not a reliable finished signal.

What you are being asked to decide

What should decide when an agent loop stops? Tests using stop_reason, not text patterns, as the loop's control signal.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
stop_reasonthe response field that says tool_use (keep going) or end_turn (done)
Agent loopthe code that calls the model, runs tools and decides whether to call again

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object stop_reason Agent loop Stop reason

Understand Q223's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Both failures come from reading completion out of text. The response's stop_reason is the structured answer: continue on tool_use, stop on end_turn, and keep the turn cap only as a safety ceiling.

Connect the design to the situation

A support loop exits on a phrase that customers sometimes write themselves.

Best answer: B. The Messages API already reports why each turn ended: tool_use means run the requested tools and continue, end_turn means the model is done. A rarer or better-anchored sentinel still guesses at completion from wording, so one of the two failures survives.

What each choice would do here

A · Why this does not meet the full requirement

A rarer sentinel such as a random token would fix both failures, because customers could never type it by accident.

A rarer phrase still depends on the model choosing to emit it.

An unusual token solves the accidental-quote case but not the other one: the model can finish its work without printing the token, and the loop then runs to the cap. It is still inferring completion from prose.

B · Why this meets the requirement

Every response already carries a stop_reason that separates wanting a tool (tool_use) from being finished (end_turn), so the loop should branch on that field rather than on text.

Right: the API signals completion structurally; text matching is a guess.

Branching on stop_reason removes both failures: quoted phrases no longer end the loop, and a finished turn ends it immediately instead of spinning to the cap. Other values such as max_tokens get their own handling.

C · Why this does not meet the full requirement

Keep the sentinel but normalise case and require it at the very end of the message, so quoted text cannot trigger it.

Tighter matching narrows one failure but keeps completion tied to wording.

Anchoring the phrase reduces false triggers from quotes, yet any run where the model finishes without the exact ending still loops to the cap. The loop is still reading prose when a structured field exists.

D · Why this does not meet the full requirement

The 25-turn cap is the real defect; raising it lets long tickets reach the sentinel before the loop gives up.

Raising the cap makes the over-running case worse and leaves the false stops untouched.

Tickets that already finished would spin even longer, costing tokens and latency. The early exits caused by quoted text would still happen. A safety ceiling is fine, but it is not the completion signal.

Check your understanding · explain the design in your own words

What should decide when an agent loop stops?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object stop_reason Agent loop Stop reason

Return to Q223's situation and clause-by-clause explanation

Q224. A travel-planning coordinator runs a flight agent and then a hotel agent. The flight agent finds that the only affordable flight lands a day later than the traveller asked for and returns that itinerary. The hotel agent then books rooms starting on the originally requested date, leaving a paid night unused. The two agents never exchange messages. Which change fixes this at the right layer? A) Add a shared table that both agents poll while they work, so each can see the other's latest decisions. B) Run the hotel agent first, so the booked dates constrain which flights the flight agent is allowed to choose. C) The coordinator puts the flight agent's confirmed arrival date and time into the hotel agent's delegation prompt. D) Give the hotel agent read access to the flight agent's working log so it can find the arrival date on its own.

Understand Q224 · the situation and each part of the question

The real situation behind this question

A travel startup sells complete trip bookings. Travellers are furious when they pay for a hotel night they cannot use because the flight lands a day later. The flight agent did its job, and so did the hotel agent, given what each was told. The failure sits in the gap between them, and the team needs to decide which component owns closing it.

Read the question in parts

Part 1 · The two agents never exchange messages

Workers are isolated spokes, so any dependency between them must travel through the coordinator.

Part 2 · at the right layer

The fix belongs where both results are visible, which is the coordinator rather than either worker.

What you are being asked to decide

How should a later agent learn what an earlier agent decided? Tests that the coordinator must inject dependent results into later delegations.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Hub-and-spokethe coordinator is the hub; the flight and hotel agents are spokes that never talk
Handoffthe hotel brief, which must include the confirmed arrival from the flight step
Context isolationeach agent sees only its own brief, not its sibling's results

Study: Coordinator handoffs: wait, pass context, then continue Study: See the handoff as data Hub-and-spoke Handoff Context isolation

Understand Q224's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Sibling workers do not see each other's results; the coordinator does. When one step's output constrains another, the coordinator must write that output into the next brief. Here, the flight's real arrival date belongs in the hotel agent's prompt.

Connect the design to the situation

A hotel is booked for a night the traveller will still be in the air.

Best answer: C. The hotel agent booked from the traveller's original request because that is all it was given. Only the coordinator sees both results, so it must forward the confirmed arrival; reordering or side channels still leave that handoff missing.

What each choice would do here

A · Why this does not meet the full requirement

Add a shared table that both agents poll while they work, so each can see the other's latest decisions.

A side channel bypasses the coordinator and adds timing races.

Polling a shared table invites the hotel agent to read a half-finished flight search, and it removes the coordinator's control of what each worker knows. The simpler fix is an explicit handoff after the flight result is final.

B · Why this does not meet the full requirement

Run the hotel agent first, so the booked dates constrain which flights the flight agent is allowed to choose.

Reordering does not create a handoff; it just moves the dependency the other way.

If hotels are booked first, the flight agent still needs those dates passed to it, and the cheapest flight may no longer be available. Sequencing is necessary but not sufficient without passing the result.

C · Why this meets the requirement

The coordinator puts the flight agent's confirmed arrival date and time into the hotel agent's delegation prompt.

Right: the dependency is carried by the hub, which is the only party that sees both results.

The hotel agent can only act on what its brief contains. Handing it the confirmed arrival makes the booking start on the right night, and it keeps the flow explicit and auditable in the coordinator.

D · Why this does not meet the full requirement

Give the hotel agent read access to the flight agent's working log so it can find the arrival date on its own.

Self-discovery works around the missing handoff and couples workers to each other's internals.

The hotel agent would have to know the log exists, parse another agent's scratch notes, and pick out the final choice from rejected options. The coordinator already holds the clean result and should simply pass it on.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

The hotel brief with the dependency carried by the coordinator

JSON
{
 "task": "Book a hotel near the conference venue",
 "constraints": {
  "check_in": "2026-11-04",
  "check_out": "2026-11-07",
  "max_nightly_rate_usd": 220
 },
 "from_flight_agent": {
  "arrival_local": "2026-11-04T21:40",
  "airport": "SIN",
  "note": "Only affordable option lands one day after the requested date"
 },
 "return_format": "hotel name, confirmation id, total cost"
}

The check-in date comes from the flight agent's confirmed arrival, not from the traveller's original request. The coordinator is the only party that sees both, so it writes the dependency into this brief.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How should a later agent learn what an earlier agent decided?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Coordinator handoffs: wait, pass context, then continue Study: See the handoff as data

Study: Coordinator handoffs: wait, pass context, then continue Study: See the handoff as data Hub-and-spoke Handoff Context isolation

Return to Q224's situation and clause-by-clause explanation

Q225. A hospital's procurement officer wants three infusion-pump vendors compared on recall history, five-year total cost of ownership and nurse usability, with every criterion measured the same way for all three vendors so the committee can read the table across. Which decomposition is MOST likely to produce a comparison that cannot be read side by side? A) One worker per criterion, each covering all three vendors, followed by a synthesis step. B) One worker per criterion, with the coordinator fixing the metric definition in each brief, then synthesis. C) One worker per vendor, each researching all three criteria for its own vendor, merged at the end. D) A single worker researching every vendor and criterion in one pass, filling a supplied comparison template.

Understand Q225 · the situation and each part of the question

The real situation behind this question

A hospital is replacing hundreds of infusion pumps, a decision with patient-safety and budget consequences. The procurement committee meets once and must compare vendors quickly. If one vendor's recall count covers five years and another's covers two, the committee could pick the less safe pump without realising the numbers were never comparable. The officer needs a research plan that guarantees aligned columns.

Read the question in parts

Part 1 · measured the same way for all three vendors

Comparability is the explicit requirement, so the decomposition must keep one definition per criterion.

Part 2 · MOST likely to produce a comparison that cannot be read side by side

The question asks for the failing design, so the answer is the split that lets metrics diverge.

What you are being asked to decide

Which way of splitting a comparison task breaks like-for-like results? Tests choosing decomposition axes that preserve comparability.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Decompositionthe choice to split by vendor or by criterion, which decides whether metrics match
Parallelizationrunning the criterion tracks at once without letting each invent its own measure

Study: Multi-agent design judgments (scenario 3 territory) Decomposition Parallelization Orchestrator

Understand Q225's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Comparability is decided by the decomposition axis. Give each criterion to one worker that covers every vendor, ideally with the metric fixed in the brief. Splitting by vendor invites three private definitions of the same measure.

Connect the design to the situation

Three pump vendors must be compared on the same three measures.

Best answer: C. Splitting by vendor lets each worker pick its own measures, so the merged columns do not line up. Splitting by criterion keeps one measure per row across all vendors, and fixing the metric in each brief makes that even stronger.

What each choice would do here

A · Why this does not meet the full requirement

One worker per criterion, each covering all three vendors, followed by a synthesis step.

This keeps each criterion's metric consistent across vendors because one worker measures it everywhere.

The recall worker applies one definition to all three vendors, and so do the cost and usability workers. Synthesis then assembles rows that are already comparable, which is exactly what the committee asked for.

B · Why this does not meet the full requirement

One worker per criterion, with the coordinator fixing the metric definition in each brief, then synthesis.

This is the strongest version of criterion-based decomposition, not a failure case.

Fixing the metric in the brief (for example, recalls per 10,000 units over five years) removes the last source of drift. It is the design that most reliably produces a table the committee can read across.

C · Why this meets the requirement

One worker per vendor, each researching all three criteria for its own vendor, merged at the end.

Right: three independent workers will each choose their own metrics, breaking comparability.

One worker may report recalls per year, another recalls per 10,000 units, and a third only class I recalls. Cost may include consumables in one track and not another. The merged table looks complete but its columns do not line up.

D · Why this does not meet the full requirement

A single worker researching every vendor and criterion in one pass, filling a supplied comparison template.

Slower and less parallel, but one worker applies one set of definitions throughout.

A single pass may take longer and strain context on large comparisons, yet its metrics stay consistent because the same worker defines them. It is less efficient, not less comparable, so it is not the design most likely to fail here.

Check your understanding · explain the design in your own words

Which way of splitting a comparison task breaks like-for-like results?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Multi-agent design judgments (scenario 3 territory)

Study: Multi-agent design judgments (scenario 3 territory) Decomposition Parallelization Orchestrator

Return to Q225's situation and clause-by-clause explanation

Q226. A platform team is choosing an orchestration style for four upcoming jobs. Which TWO jobs are better served by a fixed prompt chain than by dynamic adaptive decomposition? (Select TWO.) A) Migrating an undocumented internal library whose callers and hidden dependencies have not yet been mapped. B) A weekly compliance check that runs the same five named checks on every pull request in a fixed order. C) Investigating why a mobile app's crash rate doubled after an unknown combination of recent releases. D) Nightly processing of supplier invoices: extract fields, validate totals, post to the ledger, the same three steps every night.

Understand Q226 · the situation and each part of the question

The real situation behind this question

A platform team is standardising how it builds agentic jobs and wants a rule for when to use a simple chain versus a planner that decides subtasks at runtime. Adaptive planning costs more tokens and is harder to test, while a chain is cheap and predictable but brittle when the work is open-ended. The team lists four real jobs and asks which ones deserve the simpler design.

Read the question in parts

Part 1 · better served by a fixed prompt chain

The question is inverted: it asks where predictable structure wins, not where adaptivity is needed.

Part 2 · dynamic adaptive decomposition

The alternative style, which pays off only when subtasks cannot be known until the work begins.

What you are being asked to decide

Which jobs have steps known in advance, so a fixed chain beats adaptive planning? Tests recognising when work is predictable enough for a fixed pipeline.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Prompt chaininga fixed sequence of focused steps, right for invoices and weekly checks
Decompositionadaptive splitting that is needed when subtasks only emerge from findings

Study: A known sequence is a chain, not delegation Study: A case that does need dynamic delegation Prompt chaining Decomposition Workflow

Understand Q226's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Ask whether you can write the steps and their order before the run. If yes, as with invoices and fixed weekly checks, use a chain. If the subtasks only show up as findings come in, as with crash diagnosis and an unmapped migration, plan adaptively.

Connect the design to the situation

Four jobs: invoices, weekly checks, a crash investigation and an unmapped migration.

Best answer: BD. A chain fits when the steps and their order are known before the run starts, as with nightly invoices and a fixed set of weekly checks. Crash investigation and an unmapped migration only reveal their subtasks as findings arrive, so they call for adaptive decomposition.

What each choice would do here

A · Why this does not meet the full requirement

Migrating an undocumented internal library whose callers and hidden dependencies have not yet been mapped.

Unknown scope; the plan has to grow as callers and dependencies are discovered.

Until the callers are found, nobody knows how many subtasks exist or in what order they must run. That is exactly the situation adaptive decomposition is for, so a fixed chain would be the wrong fit.

B · Why this does not meet the full requirement

A weekly compliance check that runs the same five named checks on every pull request in a fixed order.

Right: a predictable, repeated set of checks needs no runtime planning.

Each check is known and ordered in advance, so a fixed chain or parallel set is simpler, cheaper and easier to audit. Nothing the checks find changes which checks should run next.

C · Why this does not meet the full requirement

Investigating why a mobile app's crash rate doubled after an unknown combination of recent releases.

Open-ended diagnosis; the next step depends on what each finding reveals.

The investigator does not know which release, library or device class matters until early findings narrow it down. A fixed chain would either miss the cause or waste effort on irrelevant steps, so adaptive decomposition fits.

D · Why this does not meet the full requirement

Nightly processing of supplier invoices: extract fields, validate totals, post to the ledger, the same three steps every night.

Right: known steps in a stable order are the definition of a chain.

The work never changes shape, so a chain gives predictable cost, easy testing of each step, and clear failure points. An adaptive planner would spend tokens rediscovering a plan that is already known.

Check your understanding · explain the design in your own words

Which jobs have steps known in advance, so a fixed chain beats adaptive planning?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: A known sequence is a chain, not delegation Study: A case that does need dynamic delegation

Study: A known sequence is a chain, not delegation Study: A case that does need dynamic delegation Prompt chaining Decomposition Workflow

Return to Q226's situation and clause-by-clause explanation

Q227. An analytics team inherits a data warehouse of 900 tables with no lineage documentation, and nobody knows which tables actually feed the revenue dashboards. An agent is asked to produce useful documentation for the warehouse. What should the agent's first phase be? A) Trace lineage backward from the dashboards' queries to find which tables matter, then set documentation depth according to what that map reveals. B) Document every table alphabetically with one uniform template so coverage is even and progress is easy to track. C) Write the complete documentation outline for all 900 tables in a single pass before looking at any query history. D) Split the 900 tables evenly across ten parallel workers, each documenting its 90 tables in full.

Understand Q227 · the situation and each part of the question

The real situation behind this question

An analytics team took over a warehouse from a group that left the company. Finance relies on the revenue dashboards, and an outage or misunderstanding could delay quarterly reporting. There is no lineage map and no list of critical tables. The team asked an agent to document the warehouse, and wants effort focused where it matters rather than spread thinly across hundreds of stale tables.

Read the question in parts

Part 1 · no lineage documentation

Dependencies are unknown, so any upfront plan would guess at relationships between tables.

Part 2 · nobody knows which tables actually feed the revenue dashboards

Priorities are unknown, which means discovering importance must come before deciding effort.

What you are being asked to decide

What should an agent do first when nobody knows what matters in a large unfamiliar system? Tests choosing explore-then-plan for work with unknown structure and priorities.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Decompositionsplitting the documentation job in a way that follows discovered dependencies
Progressive discoverylearning the warehouse's shape step by step before committing to a plan

Study: A case that does need dynamic delegation Decomposition Structured discovery Progressive discovery

Understand Q227's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When you do not yet know the structure or what matters, a fixed plan spreads effort blindly. Start by mapping from the business-critical outputs backward, then let the documentation plan follow what the map shows.

Connect the design to the situation

Nine hundred undocumented warehouse tables, only some of which feed revenue dashboards.

Best answer: A. No lineage and no known priorities mean a fixed plan would be guesswork. Mapping from the dashboards backward reveals which tables matter and how they connect, and the documentation plan then grows from that map. Even or alphabetical splits spend equal effort on everything.

What each choice would do here

A · Why this meets the requirement

Trace lineage backward from the dashboards' queries to find which tables matter, then set documentation depth according to what that map reveals.

Right: map first to discover priorities and dependencies, then let the plan adapt.

Starting from the dashboards reveals the small set of tables that carry business weight and how they connect. The agent can then document those deeply and treat unused tables lightly, adjusting as new dependencies appear.

B · Why this does not meet the full requirement

Document every table alphabetically with one uniform template so coverage is even and progress is easy to track.

Uniform effort spends as much on dead tables as on revenue-critical ones.

Alphabetical order ignores dependencies and importance. The agent would write the same depth for abandoned staging tables as for the tables behind revenue reporting, and would describe tables before understanding how they relate.

C · Why this does not meet the full requirement

Write the complete documentation outline for all 900 tables in a single pass before looking at any query history.

A full plan written before exploration encodes guesses as structure.

Without query history the outline cannot know which tables matter or how they connect, so it will over-plan unimportant areas and miss hidden dependencies. The plan should follow the map, not precede it.

D · Why this does not meet the full requirement

Split the 900 tables evenly across ten parallel workers, each documenting its 90 tables in full.

Parallel even splits scale the wrong plan faster.

Even splits scatter related tables across workers who never see each other's findings, and they spend equal effort regardless of importance. Parallelism is useful later, once the map shows which clusters to document together.

Check your understanding · explain the design in your own words

What should an agent do first when nobody knows what matters in a large unfamiliar system?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: A case that does need dynamic delegation

Study: A case that does need dynamic delegation Decomposition Structured discovery Progressive discovery

Return to Q227's situation and clause-by-clause explanation

Q228. A law firm's intake assistant has five specialists: conflict checking, jurisdiction research, fee estimation, document drafting and translation. Its coordinator invokes all five for every inquiry, including "What are your office hours?", and the partners complain about cost and slow replies. Where should the fix live? A) In the coordinator: assess each inquiry and dispatch only the specialists it actually needs. B) In the front end: add a separate FAQ bot so simple questions never reach the multi-agent system. C) In the specialists' configuration: lower each one's output budget so unnecessary runs finish faster. D) In each specialist: add an early check that returns "not applicable" when an inquiry is outside its area.

Understand Q228 · the situation and each part of the question

The real situation behind this question

A mid-size law firm launched an intake assistant to handle website inquiries around the clock. Most inquiries are simple: hours, parking, whether the firm handles a type of case. A few are real new matters that need conflicts checked and fees estimated. Every inquiry currently runs the full specialist set, so the bill is high and simple questions wait many seconds for an answer.

Read the question in parts

Part 1 · invokes all five for every inquiry

The coordinator uses a fixed full pipeline, which is the design flaw to correct.

Part 2 · Where should the fix live

The question asks for the right component to own the change, not just any change that lowers cost.

What you are being asked to decide

Who should decide which specialists a request needs? Tests putting per-request specialist selection in the coordinator.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Workflow routingclassifying each inquiry and sending it only to the specialists it needs
Orchestratorthe intake coordinator, which sees each whole inquiry and can make that call once

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design Workflow routing Orchestrator Latency

Understand Q228's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Deciding which specialists a request needs is a routing decision, and the coordinator is the only component that sees the whole request. Have it classify each inquiry and dispatch only the specialists required, so simple questions stay cheap and complex ones keep full coverage.

Connect the design to the situation

A simple office-hours question triggers five legal specialists.

Best answer: A. Running five specialists for an office-hours question is a routing failure, and routing belongs to the coordinator, which sees the whole inquiry. Specialists that decline, a second bot, or smaller budgets all keep paying for work that should never start.

What each choice would do here

A · Why this meets the requirement

In the coordinator: assess each inquiry and dispatch only the specialists it actually needs.

Right: routing is the coordinator's job, and it removes unnecessary invocations entirely.

An office-hours question then goes to no specialist or one, while a new-matter inquiry still gets conflicts, jurisdiction and fees. Cost and latency fall for simple inquiries without weakening complex ones.

B · Why this does not meet the full requirement

In the front end: add a separate FAQ bot so simple questions never reach the multi-agent system.

A second system duplicates routing logic and maintenance for a problem one coordinator can solve.

Now two systems must agree on what counts as simple, and borderline inquiries fall between them. The existing coordinator already sees every inquiry and can make the same decision with less machinery.

C · Why this does not meet the full requirement

In the specialists' configuration: lower each one's output budget so unnecessary runs finish faster.

Smaller budgets shave cost per run but keep every unnecessary run.

The office-hours question would still trigger five specialists, just with shorter answers, and the needed specialists on complex matters might now be truncated. It treats the symptom, not the routing problem.

D · Why this does not meet the full requirement

In each specialist: add an early check that returns "not applicable" when an inquiry is outside its area.

The specialists still start, load context and consume a turn before declining.

Five spawns per inquiry remain, each paying startup cost just to say no. It spreads a routing decision across five components instead of making it once where the whole request is visible.

Check your understanding · explain the design in your own words

Who should decide which specialists a request needs?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design Workflow routing Orchestrator Latency

Return to Q228's situation and clause-by-clause explanation

Q229. A security analyst has spent a long session triaging an intrusion and building a detailed attack timeline. She now wants to evaluate two containment plans (isolate the affected subnet, or rotate every credential and keep services running), each reasoned from the same full timeline, without either plan's reasoning leaking into the other. What should she do? A) Export the timeline to a notes file and open two new sessions that each read the file. B) Resume the session, evaluate the subnet plan, then clear the conversation and evaluate the credential plan. C) Fork the session from the finished timeline twice and evaluate one containment plan in each branch. D) Ask the agent to evaluate both plans side by side in a single reply so neither one is favoured.

Understand Q229 · the situation and each part of the question

The real situation behind this question

During an active intrusion, the security lead must choose between two disruptive containment plans. Isolating the subnet stops lateral movement but takes customer services down; rotating credentials keeps services running but may miss a persistence mechanism. The analyst's timeline is the product of hours of log reading. She needs each plan judged fairly, on the same evidence, before the lead decides.

Read the question in parts

Part 1 · each reasoned from the same full timeline

Both evaluations need the complete baseline, which rules out selective summaries.

Part 2 · without either plan's reasoning leaking into the other

Independence is required, which rules out doing both in one context or sequentially.

What you are being asked to decide

How do you test two alternatives from the same analysis without mixing them? Tests using a session fork to branch from a shared baseline.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Checkpointthe finished timeline, the point in the session from which both branches start
Context isolationkeeping the subnet plan's reasoning out of the credential plan's branch

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names Checkpoint Context isolation Agent SDK

Understand Q229's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Two alternatives, one shared baseline, no cross-talk: that is what a fork is for. Branch the session after the timeline is complete and evaluate one plan per branch.

Connect the design to the situation

Two containment plans must each be reasoned from one detailed intrusion timeline.

Best answer: C. Both evaluations need the same complete baseline and must stay isolated from each other. Forking copies the full session history into each branch; a notes export keeps only a selective summary, and working in one context lets each plan's reasoning shape the other.

What each choice would do here

A · Why this does not meet the full requirement

Export the timeline to a notes file and open two new sessions that each read the file.

A written summary drops evidence and tool results the full session holds.

The notes preserve conclusions but not the log lines, indicators and rejected hypotheses behind them. Exporting is a fallback when forking is unavailable, but here it trades away detail the analyst needs.

B · Why this does not meet the full requirement

Resume the session, evaluate the subnet plan, then clear the conversation and evaluate the credential plan.

Clearing the conversation also discards the timeline the second plan needs.

After a clear, the second evaluation starts without the detailed evidence built during triage. The analyst would either rebuild it or reason from memory, so the two plans no longer share a baseline.

C · Why this meets the requirement

Fork the session from the finished timeline twice and evaluate one containment plan in each branch.

Right: each fork starts with the full timeline, and later turns stay independent.

Both branches inherit every tool result and finding that produced the timeline, and neither sees what the other concludes. That gives two clean evaluations from an identical starting point.

D · Why this does not meet the full requirement

Ask the agent to evaluate both plans side by side in a single reply so neither one is favoured.

One reply means one context: each plan's reasoning shapes the other.

Writing both evaluations together invites the model to frame the second plan against the first, carrying assumptions across. It is fast but gives up the independence the analyst explicitly wanted.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Forking one saved session into two branches (CLI)

TEXT
# baseline session id comes from the triage run
claude --resume 7c1e-triage --fork-session -p "Evaluate containment plan one: isolate the affected subnet."
claude --resume 7c1e-triage --fork-session -p "Evaluate containment plan two: rotate all credentials, keep services running."

Each command resumes the same saved triage session but forks it, so each run gets a new session id whose history starts as a copy of the timeline. Neither branch writes into the original or sees the other.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you test two alternatives from the same analysis without mixing them?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names Checkpoint Context isolation Agent SDK

Return to Q229's situation and clause-by-clause explanation

Q230. During a long session tuning a pricing service, operations roll back a deployment and several configuration files on disk change. The agent keeps quoting the old values it read an hour ago. A teammate says, "Just tell it to reread the changed files." Why is that alone unlikely to fully fix the problem? A) The context window is already full, so any new file contents are silently dropped before the model sees them. B) Rereading works fully; the agent simply needs extended thinking enabled to notice that the values changed. C) The earlier reads stay in the conversation beside the new ones, so the agent can still anchor on the outdated values in later reasoning. D) File tools cache results for the lifetime of a session, so a reread returns the same stale content.

Understand Q230 · the situation and each part of the question

The real situation behind this question

A pricing engineer has spent an afternoon with an agent tuning discount rules. A production incident forced operations to roll back the latest deployment, which reverted several configuration files. The agent is now recommending changes against values that no longer exist, which could ship a pricing mistake to customers. The team wants to understand why a quick reread might not be enough before choosing a recovery.

Read the question in parts

Part 1 · keeps quoting the old values it read an hour ago

Stale tool results are still active in context and still shaping answers.

Part 2 · Why is that alone unlikely to fully fix the problem

The question asks for the mechanism behind the failure, not just a recovery step.

What you are being asked to decide

Why doesn't rereading changed files fully cure stale context? Tests understanding that stale tool results persist in history and how to recover.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Messages arraythe growing history where the old file reads still sit after a reread
Context windoweverything the model sees each turn, now containing two versions of each file

Study: Fork, resume, or start fresh? Context window Compaction Messages array

Understand Q230's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Tool results stay in the conversation, so rereading adds current values without removing old ones. When stale reads are widespread, start a fresh session seeded with the decisions that still hold and read the current files there; a fork would copy the stale history along.

Connect the design to the situation

A rollback changed config files an agent read an hour earlier.

Best answer: C. A reread appends current file contents while the stale reads, and everything reasoned from them, stay in context. The agent can keep anchoring on the old values. Starting fresh with a summary of the decisions that still hold, then reading the current files, removes the conflict.

What each choice would do here

A · Why this does not meet the full requirement

The context window is already full, so any new file contents are silently dropped before the model sees them.

A full window produces errors or compaction, not silent dropping of new tool results.

If the conversation were at its limit, the request would fail or the runtime would compact older content. Nothing in the situation suggests that, and rereading would still deliver current contents to the model.

B · Why this does not meet the full requirement

Rereading works fully; the agent simply needs extended thinking enabled to notice that the values changed.

More reasoning does not remove conflicting evidence from context.

Extended thinking might help the agent reconcile the two versions sometimes, but it still reasons over a history full of stale reads and conclusions drawn from them. Removing the stale material is more reliable than hoping the model sorts it out.

C · Why this meets the requirement

The earlier reads stay in the conversation beside the new ones, so the agent can still anchor on the outdated values in later reasoning.

Right: rereading appends; it does not remove the stale observations.

The context now holds two versions of the same files, and older content has been reinforced by an hour of reasoning built on it. A fresh session carrying a summary of still-valid decisions, followed by current reads, removes the conflict.

D · Why this does not meet the full requirement

File tools cache results for the lifetime of a session, so a reread returns the same stale content.

A reread returns current disk contents; there is no session-long file cache.

The read tool fetches the file as it is now, so the new values do arrive. The trouble is that the old values also remain in history, not that the reread is served from a cache.

Check your understanding · explain the design in your own words

Why doesn't rereading changed files fully cure stale context?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Fork, resume, or start fresh?

Study: Fork, resume, or start fresh? Context window Compaction Messages array

Return to Q230's situation and clause-by-clause explanation

Q231. A due-diligence system produces a memo on an acquisition target. The coordinator's review finds that one revenue figure has no source and the question about pending litigation was never answered; everything else is solid. Which next step keeps the good work and closes the holes? A) Ask the synthesis worker to supply the revenue source and litigation status from what it already knows. B) Send narrowly scoped follow-up tasks for the unsourced revenue figure and the litigation question, then rerun synthesis with the new findings. C) Discard the memo and rerun every worker with a longer, more detailed brief. D) Mark the two gaps as unverified and deliver the memo, letting the client's analysts chase the missing evidence.

Understand Q231 · the situation and each part of the question

The real situation behind this question

A private-equity firm uses the system to screen acquisition targets before committing lawyers and bankers. A memo with an unsourced revenue figure or a missing litigation check could lead to a bad bid. Most of this memo is solid, and the deal team wants it quickly. The coordinator must close two holes without delaying everything or letting unsupported claims through.

Read the question in parts

Part 1 · one revenue figure has no source

A specific evidence gap that needs new retrieval, not rewording.

Part 2 · everything else is solid

The verified sections should be preserved, which argues against a full restart.

What you are being asked to decide

What does the coordinator do when a mostly good synthesis has two specific gaps? Tests targeted re-delegation to fill gaps before re-synthesis.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Provenanceeach claim in the memo must point to a source, which the revenue figure lacks
Hallucination vs retrievalasking synthesis to fill gaps from memory invites invention; follow-up research retrieves

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report Evaluator-optimizer Provenance Hallucination vs retrieval

Understand Q231's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Specific gaps need specific evidence. The coordinator sends targeted follow-up research to workers that can retrieve it, then re-runs synthesis, keeping everything that was already verified.

Connect the design to the situation

An acquisition memo lacks a source for revenue and an answer on litigation.

Best answer: B. Two specific holes call for two specific follow-ups, not a restart and not an answer from a synthesis step that has no way to search. Re-delegating targeted research and then re-running synthesis keeps the verified work and fills the gaps with evidence.

What each choice would do here

A · Why this does not meet the full requirement

Ask the synthesis worker to supply the revenue source and litigation status from what it already knows.

Synthesis has no research tools, so this invites invented citations.

The synthesis worker only has the findings it was given. Asking it to fill evidence gaps pushes it toward plausible-sounding sources and a guessed litigation status, which is the worst outcome in due diligence.

B · Why this meets the requirement

Send narrowly scoped follow-up tasks for the unsourced revenue figure and the litigation question, then rerun synthesis with the new findings.

Right: targeted re-delegation fills exactly the gaps and preserves the rest.

Research workers with search and filing access can find the revenue source and court records. Synthesis then updates the memo with evidence instead of guesses, and the verified sections are left untouched.

C · Why this does not meet the full requirement

Discard the memo and rerun every worker with a longer, more detailed brief.

A full restart throws away verified work to fix two specific holes.

Most of the memo is sound. Rerunning everything costs time and tokens, risks new inconsistencies in sections that were fine, and may still leave the same two gaps if the briefs do not target them.

D · Why this does not meet the full requirement

Mark the two gaps as unverified and deliver the memo, letting the client's analysts chase the missing evidence.

Flagging gaps is honest but pushes the system's job onto the client.

Labelling the holes avoids false claims, which is better than inventing them, but the coordinator has workers that can close these gaps now. Delivering incomplete diligence shifts risk to the buyer without need.

Check your understanding · explain the design in your own words

What does the coordinator do when a mostly good synthesis has two specific gaps?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report Evaluator-optimizer Provenance Hallucination vs retrieval

Return to Q231's situation and clause-by-clause explanation

Q232. An insurance intake agent is meant only to collect and verify policy details, but it has eleven tools, including approve_payout, draft_denial_letter and score_fraud_risk. In testing it sometimes drafts denial letters during intake. A system-prompt warning has already been tried. Which fix removes the cause rather than the symptom? A) Give the intake agent only its collection and verification tools, and move payout, denial and fraud tools to the adjudication agent. B) Strengthen the system prompt with a bold rule that the intake agent must never use adjudication tools. C) Keep all eleven tools but add a hook that rejects adjudication tool calls coming from the intake agent. D) Rewrite the adjudication tools' descriptions to say they are only for use after intake is complete.

Understand Q232 · the situation and each part of the question

The real situation behind this question

An insurer is automating first notice of loss. The intake agent should gather and verify details, then hand off to adjudication. In testing it occasionally drafted denial letters on day one of a claim, which would be a regulatory and reputational disaster if it reached a customer. The team already tried a stern instruction and wants a fix that addresses why this happens.

Read the question in parts

Part 1 · eleven tools

The agent's menu is much larger than its role, which is the underlying cause.

Part 2 · removes the cause rather than the symptom

The answer must change what the agent can choose, not just catch or discourage the wrong choice.

What you are being asked to decide

How do you stop an agent from using tools that belong to another role? Tests scoping each agent's tool menu to its role.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Least-privilege toolinggiving the intake agent only collection and verification tools
Capability bloateleven tools on an agent whose job needs three or four

Study: Decision rule: scope the menu to the role, then enforce it in code Study: A tool menu is not an execution sequence Least-privilege tooling Tool selection Capability bloat

Understand Q232's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The number and kind of tools an agent sees shape what it does. Give the intake agent only its intake tools and let adjudication own the rest; keep hooks as a backstop, not as the primary fix.

Connect the design to the situation

An intake agent drafts denial letters because it can see a denial tool.

Best answer: A. The intake agent drafts denials because it can see a tool for it. A prompt warning has already failed, and a hook only rejects the call after the wrong choice is made. Scoping the menu to the role removes the choice itself.

What each choice would do here

A · Why this meets the requirement

Give the intake agent only its collection and verification tools, and move payout, denial and fraud tools to the adjudication agent.

Right: removing out-of-role tools removes the wrong choices from the menu.

With only collection and verification tools visible, the intake agent cannot choose to draft a denial, and its tool selection gets easier. The adjudication agent keeps the tools where they belong.

B · Why this does not meet the full requirement

Strengthen the system prompt with a bold rule that the intake agent must never use adjudication tools.

A prompt rule was already tried; the tools remain visible and callable.

Instructions compete with tool descriptions the model sees every turn. Having already failed once, a louder version is still guidance rather than a structural change, and the tools stay on the menu.

C · Why this does not meet the full requirement

Keep all eleven tools but add a hook that rejects adjudication tool calls coming from the intake agent.

A hook blocks the call after the model has already chosen wrongly.

Blocking prevents damage, which is useful as defence in depth, but the agent still spends turns choosing forbidden tools and then recovering from rejections. The menu still invites the wrong action.

D · Why this does not meet the full requirement

Rewrite the adjudication tools' descriptions to say they are only for use after intake is complete.

Better descriptions help, but the tools remain on a menu they should not be on.

Clearer descriptions might reduce misuse somewhat, yet the intake agent still sees and can call approve_payout. The cleanest fix for a role mismatch is to take those tools off its menu entirely.

Check your understanding · explain the design in your own words

How do you stop an agent from using tools that belong to another role?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Decision rule: scope the menu to the role, then enforce it in code Study: A tool menu is not an execution sequence

Study: Decision rule: scope the menu to the role, then enforce it in code Study: A tool menu is not an execution sequence Least-privilege tooling Tool selection Capability bloat

Return to Q232's situation and clause-by-clause explanation

Q233. A release pipeline uses three generator subagents: one writes the iOS client, one the Android client, and one the shared API contract. After a run, Android sends timestamps as epoch milliseconds, iOS sends ISO strings, and the contract expects ISO strings. Each generator's output is internally consistent. Where should the consistency check live? A) In the coordinator's own planning turn, with extended thinking enabled so it can compare the three outputs itself. B) In a dedicated reviewer that receives all three outputs from the coordinator and checks only cross-component agreement. C) In each generator, by letting it browse the other generators' working directories while it runs. D) In each generator, re-prompted with an instruction to match the other platforms' formats before returning.

Understand Q233 · the situation and each part of the question

The real situation behind this question

A mobile team generates both native clients and the API contract with separate subagents to move fast. Each output looks fine on its own, but integration keeps failing on small mismatches like timestamp formats. Every failure costs a slow integration run and a debugging session. The team wants mismatches caught right after generation, and needs to decide which component should own that check.

Read the question in parts

Part 1 · Each generator's output is internally consistent

Individual generators are not the problem; the failures are between them.

Part 2 · Where should the consistency check live

The question asks which component owns the check, which must be one that sees all outputs.

What you are being asked to decide

Where should you check that separately generated parts agree with each other? Tests adding a dedicated cross-output validation step.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Hub-and-spokegenerators never see each other, so the coordinator must gather outputs for any comparison
Context isolationwhy no single generator can notice the mismatch on its own

Study: Diagnose the earliest evidenced failure Study: What does single-purpose mean? Subagent Hub-and-spoke Context isolation

Understand Q233's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Cross-component bugs need a check that sees all components. Have the coordinator pass every output to a single-purpose reviewer that looks only for disagreements, so mismatches are caught before integration.

Connect the design to the situation

iOS and Android clients disagree on timestamp format with each other and the API.

Best answer: B. Each generator was right in isolation, and every error sits between components. Only a step that receives all three outputs can check agreement, and giving that check its own reviewer keeps it focused and out of the coordinator's planning context.

What each choice would do here

A · Why this does not meet the full requirement

In the coordinator's own planning turn, with extended thinking enabled so it can compare the three outputs itself.

The coordinator can do this, but mixing review into planning dilutes both.

Loading three full code outputs into the coordinator crowds its context and adds a detailed review to a role meant for routing. A separate reviewer with a single purpose is more focused and easier to test.

B · Why this meets the requirement

In a dedicated reviewer that receives all three outputs from the coordinator and checks only cross-component agreement.

Right: only a step that sees every output together can check agreement between them.

A reviewer with one job compares field names, formats and endpoints across the three outputs. It catches the timestamp mismatch before integration tests, and its focused brief makes it more reliable than a side task.

C · Why this does not meet the full requirement

In each generator, by letting it browse the other generators' working directories while it runs.

Workers reading each other's in-progress files creates races and coupling.

The generators run concurrently, so each would see half-finished files from the others. Consistency would depend on timing, and responsibility for the check would be spread across three agents instead of one.

D · Why this does not meet the full requirement

In each generator, re-prompted with an instruction to match the other platforms' formats before returning.

No generator can see the others' output, so the instruction cannot be followed.

Telling the Android generator to match iOS is meaningless when it has never seen the iOS code. Each generator will simply reassert its own assumptions and report that it is consistent.

Check your understanding · explain the design in your own words

Where should you check that separately generated parts agree with each other?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Diagnose the earliest evidenced failure Study: What does single-purpose mean?

Study: Diagnose the earliest evidenced failure Study: What does single-purpose mean? Subagent Hub-and-spoke Context isolation

Return to Q233's situation and clause-by-clause explanation

Q234. A database team's coordinator runs a migration-writer subagent and then a rollback-writer subagent for each ticket. Reviewers notice the rollback scripts reliably undo what each ticket describes, but miss extra changes the migration writer actually made, such as an index it added for performance. What explains the pattern? A) The tickets are too vague; more precise tickets would keep both scripts in agreement. B) The rollback writer lacks a database connection to inspect the live schema after the migration runs. C) The two writers run on different model tiers and therefore interpret the ticket differently. D) The rollback writer receives only the ticket, never the migration script the first writer produced.

Understand Q234 · the situation and each part of the question

The real situation behind this question

A database platform team automates schema changes. Each ticket gets a migration and a matching rollback, and the rollback is the safety net if a deploy goes wrong. Reviewers discovered rollbacks that would leave behind indexes and columns the migration quietly added. In an incident, an incomplete rollback could leave production in an unknown state, so the team wants to find the root cause.

Read the question in parts

Part 1 · reliably undo what each ticket describes

The rollbacks track the ticket, which reveals what the rollback writer is using as its input.

Part 2 · extra changes the migration writer actually made

Changes beyond the ticket are visible only in the migration output, which the rollback writer lacks.

What you are being asked to decide

Why do rollback scripts match the ticket but not the actual migration? Tests that dependent workers need the real upstream artefact, not just the spec.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Handoffthe rollback brief, which must contain the generated migration, not only the ticket
Orchestratorthe coordinator that holds the migration and must forward it

Study: Coordinator handoffs: wait, pass context, then continue Study: Why running sequentially is necessary here but not sufficient Handoff Subagent Orchestrator

Understand Q234's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Running steps in order is not enough; the later step must receive the earlier step's actual output. Put the generated migration in the rollback writer's brief so it reverses what was really done.

Connect the design to the situation

Rollbacks miss an index the migration writer added beyond the ticket.

Best answer: D. The rollbacks follow the ticket because the ticket is all the rollback writer ever saw; it never received the migration. Better tickets or matched models cannot expose changes that only exist in the migration script, so the coordinator must pass the actual migration into the rollback brief.

What each choice would do here

A · Why this does not meet the full requirement

The tickets are too vague; more precise tickets would keep both scripts in agreement.

Better tickets help planning but cannot capture changes the migration writer adds on its own.

The migration writer legitimately adds things like performance indexes that no ticket mentions. However precise the ticket, the rollback writer still reverses the ticket, not the migration, unless it sees the migration.

B · Why this does not meet the full requirement

The rollback writer lacks a database connection to inspect the live schema after the migration runs.

The rollback is written before deployment, and the needed input already exists as the migration script.

A live connection is unnecessary and risky here. The coordinator holds the exact migration text, which lists every change. Handing that over is simpler and safer than having the rollback writer query production.

C · Why this does not meet the full requirement

The two writers run on different model tiers and therefore interpret the ticket differently.

A tier difference would produce varied mismatches, not a consistent blind spot for extra changes.

The pattern is systematic: rollbacks match the ticket and miss additions beyond it. That points to what the rollback writer is given, not how capable it is. Aligning models would not reveal the unseen index.

D · Why this meets the requirement

The rollback writer receives only the ticket, never the migration script the first writer produced.

Right: it can only reverse what it was shown, which is the ticket.

The rollback matches the ticket exactly because the ticket is its only input. Any extra change the migration writer made is invisible to it, so the fix is passing the generated migration into the rollback brief.

Check your understanding · explain the design in your own words

Why do rollback scripts match the ticket but not the actual migration?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Coordinator handoffs: wait, pass context, then continue Study: Why running sequentially is necessary here but not sufficient

Study: Coordinator handoffs: wait, pass context, then continue Study: Why running sequentially is necessary here but not sufficient Handoff Subagent Orchestrator

Return to Q234's situation and clause-by-clause explanation

Q235. An IT help-desk agent runs a fourteen-question diagnostic interview (operating system, network, recent updates, log excerpts) on every ticket, including "Please reset my password". Staff rate the assistant as exhausting, yet for real outages the full interview is what engineers need. Which change serves both kinds of ticket? A) Triage each ticket by type first and pick the interview depth from that type: none for a password reset, the full diagnostic for an outage. B) Shorten the wording of each of the fourteen questions so the interview is quicker to answer. C) Drop the interview and require staff to complete a detailed form before any ticket is accepted. D) Cap the interview at four questions for every ticket.

Understand Q235 · the situation and each part of the question

The real situation behind this question

A company rolled out an AI help desk to cut ticket times. Employees now complain it interrogates them over trivial requests, and adoption is dropping. At the same time, the infrastructure team says the detailed interview is exactly what makes outage tickets actionable. The help-desk lead needs one change that reduces friction without losing the diagnostic depth engineers value, and without asking staff to change how they file tickets.

Read the question in parts

Part 1 · on every ticket

One flow is applied regardless of type, which is the root of the friction.

Part 2 · serves both kinds of ticket

The fix must keep deep questioning for outages while removing it for simple requests.

What you are being asked to decide

How do you keep clarification light for simple tickets and deep for hard ones? Tests classifying requests before choosing processing depth.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Workflow routingsorting tickets by type so each goes to a flow of the right depth
Requirement gatheringthe diagnostic interview, whose depth should match the ticket

Study: The five workflow patterns (parallelization comes in two flavors) Workflow routing Requirement gathering Workflow

Understand Q235's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When requests vary widely, classify first and route each type to the right depth of processing. A single flow, capped or not, will always be wrong for one end of the range.

Connect the design to the situation

A password reset triggers the same fourteen-question interview as an outage.

Best answer: A. Staff are frustrated because one heavy interview runs on every ticket, whatever its type. A global cap under-serves outages, a form moves the friction to the start, and shorter questions keep the same count. Classifying first lets each ticket get the depth it needs.

What each choice would do here

A · Why this meets the requirement

Triage each ticket by type first and pick the interview depth from that type: none for a password reset, the full diagnostic for an outage.

Right: classification lets each ticket get the depth it actually needs.

A password reset goes straight to the reset flow, a printer issue gets a short interview, and an outage gets all fourteen questions. Friction drops for simple tickets without starving engineers of detail on complex ones.

B · Why this does not meet the full requirement

Shorten the wording of each of the fourteen questions so the interview is quicker to answer.

Shorter questions reduce reading time but keep the wrong number of questions.

A password reset still triggers fourteen questions, just terser ones. The problem is that one flow is applied to every ticket, not that individual questions are too long.

C · Why this does not meet the full requirement

Drop the interview and require staff to complete a detailed form before any ticket is accepted.

This moves all the friction to the start and applies it to every ticket.

Every employee now fills in a long form, even for a password reset, and ambiguous outage reports still arrive without the right detail. It removes adaptive questioning where it helps most.

D · Why this does not meet the full requirement

Cap the interview at four questions for every ticket.

A uniform cap still over-asks simple tickets and under-asks outages.

A password reset still gets four unnecessary questions, while an outage loses ten questions engineers rely on. The cap trades one mismatch for another because it ignores the type of ticket.

Check your understanding · explain the design in your own words

How do you keep clarification light for simple tickets and deep for hard ones?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The five workflow patterns (parallelization comes in two flavors)

Study: The five workflow patterns (parallelization comes in two flavors) Workflow routing Requirement gathering Workflow

Return to Q235's situation and clause-by-clause explanation

Q236. An agent must write a SQL query whose output matches a finance team's reference report. On each retry it writes a brand-new query from scratch, and after eight attempts the mismatch persists in different places each time. What input is the retry loop missing? A) A system-prompt reminder to fix the previous attempt rather than start over. B) A higher retry ceiling so the agent has more attempts to land on a matching query. C) A lower temperature so each fresh attempt is more deterministic. D) The previous query together with the specific rows where its output differed from the reference.

Understand Q236 · the situation and each part of the question

The real situation behind this question

A finance team wants to automate a monthly revenue report that analysts currently build by hand. The agent's query must reproduce their reference numbers exactly before anyone will trust it. After eight attempts the numbers are still wrong, and each attempt is wrong in a new way. The team is burning budget and needs to understand why the loop is not getting closer.

Read the question in parts

Part 1 · writes a brand-new query from scratch

Each attempt ignores the previous one, which shows the loop does not carry prior work forward.

Part 2 · mismatch persists in different places each time

Errors move around rather than shrink, the signature of blind regeneration.

What you are being asked to decide

What must each retry receive for an iterative fix loop to converge? Tests carrying the prior attempt and failure details into the next iteration.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Evaluator-optimizerone step produces a query, another compares it to the reference and feeds back the diff
Feedback loop architecturedesigning each iteration to receive the last attempt and exactly what failed

Study: Evaluator-optimizer: follow one draft Evaluator-optimizer Feedback loop architecture Messages array

Understand Q236's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A loop only converges when each iteration can see what it tried and exactly how that failed. Pass the previous query and the mismatching rows into the next attempt, so the agent repairs instead of restarting.

Connect the design to the situation

A SQL-writing agent restarts from scratch on every retry and never matches the report.

Best answer: D. The agent rewrites from scratch because each retry starts with no memory of the last query or of where it went wrong. Feeding in the previous query and the rows that differed turns the loop into repair. More retries, lower temperature or a reminder cannot supply information that is not in the prompt.

What each choice would do here

A · Why this does not meet the full requirement

A system-prompt reminder to fix the previous attempt rather than start over.

The agent cannot fix an attempt it has not been shown.

If the prior query and its failures are not in the prompt, the instruction has nothing to act on. The agent would comply in spirit by writing another new query that it believes is an improvement.

B · Why this does not meet the full requirement

A higher retry ceiling so the agent has more attempts to land on a matching query.

More blind attempts do not converge any faster.

Each attempt is independent of the last, so additional retries just sample more random queries. Without knowing what went wrong before, the agent has no reason to get closer on the ninth try.

C · Why this does not meet the full requirement

A lower temperature so each fresh attempt is more deterministic.

Determinism makes attempts repeat, not improve.

At low temperature the agent may produce the same wrong query repeatedly. The problem is missing feedback, not randomness, so reducing variation can make convergence even less likely.

D · Why this meets the requirement

The previous query together with the specific rows where its output differed from the reference.

Right: the prior attempt plus the concrete diff turns regeneration into targeted repair.

With the last query and the mismatching rows in front of it, the agent can see that, say, refunds were double-counted and adjust one join. Each iteration builds on the last instead of starting over.

Check your understanding · explain the design in your own words

What must each retry receive for an iterative fix loop to converge?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Evaluator-optimizer: follow one draft

Study: Evaluator-optimizer: follow one draft Evaluator-optimizer Feedback loop architecture Messages array

Return to Q236's situation and clause-by-clause explanation

Q237. An e-commerce operations assistant spawns inventory, pricing and fraud specialists for every request. Asking "How many units of the blue kettle are in stock?" takes as long and costs as much as "Why did our margin drop last week?", which genuinely needs all three. Which statement about the redesign is exactly right? A) The cheapest fix is to keep all three specialists on every request but have them share one cached context. B) Because the margin investigation needs all three specialists, the stock lookup must also run them to keep output formats consistent. C) The coordinator should improvise a fresh architecture for each request, since no single policy can cover both kinds. D) The selection rule is designed once into the coordinator, and each request is assessed so the rule decides which specialists that request actually runs.

Understand Q237 · the situation and each part of the question

The real situation behind this question

An online retailer's operations staff use the assistant dozens of times a day. Most questions are simple lookups; a few are real investigations that need inventory, pricing and fraud analysis together. Simple lookups currently take many seconds and cost as much as an investigation. The platform lead wants a redesign that stays predictable enough to test and audit while fitting each request.

Read the question in parts

Part 1 · spawns inventory, pricing and fraud specialists for every request

The current design runs a fixed full pipeline, which is what needs to change.

Part 2 · Which statement about the redesign is exactly right

The question tests the precise relationship between fixed policy and per-request behaviour.

What you are being asked to decide

Is specialist selection a fixed design or a per-request decision? Tests separating a fixed routing policy from per-request dispatch.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Workflow routingthe fixed rule that maps an assessed request to the specialists it needs
Orchestratorthe coordinator that assesses each request and applies that rule

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design Workflow routing Orchestrator Subagent

Understand Q237's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Design the routing policy once, in the coordinator, and apply it to every request. The policy is fixed, the branch it takes is decided per request, so simple lookups stay cheap and investigations stay thorough.

Connect the design to the situation

A stock lookup runs the same three specialists as a margin investigation.

Best answer: D. The rule should be fixed and designed into the coordinator, with each request assessed so the rule picks only the specialists it needs. Improvising a new structure per request makes behaviour unpredictable, and caching still runs all three specialists.

What each choice would do here

A · Why this does not meet the full requirement

The cheapest fix is to keep all three specialists on every request but have them share one cached context.

Sharing context still runs every specialist on every request.

A shared cache might trim some input cost, but three specialists still start, reason and respond for a one-line stock question. It also breaks the isolation that keeps each specialist focused.

B · Why this does not meet the full requirement

Because the margin investigation needs all three specialists, the stock lookup must also run them to keep output formats consistent.

Output consistency does not require running unneeded specialists.

The response format can be standardised by the coordinator regardless of how many specialists contributed. Running pricing and fraud for a stock count adds cost and delay without adding information.

C · Why this does not meet the full requirement

The coordinator should improvise a fresh architecture for each request, since no single policy can cover both kinds.

Improvised architecture per request is unpredictable and hard to test.

Letting the coordinator invent a new structure every time makes cost and behaviour vary unpredictably. One stable policy can easily cover both kinds of request by selecting specialists based on assessment.

D · Why this meets the requirement

The selection rule is designed once into the coordinator, and each request is assessed so the rule decides which specialists that request actually runs.

Right: fixed policy, per-request dispatch.

The coordinator applies the same rule to every request: a stock lookup goes to inventory alone, a margin investigation goes to all three. Behaviour is predictable and testable, yet each request pays only for what it needs.

Check your understanding · explain the design in your own words

Is specialist selection a fixed design or a per-request decision?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design

Study: The five workflow patterns (parallelization comes in two flavors) Study: Choosing the simplest sufficient design Workflow routing Orchestrator Subagent

Return to Q237's situation and clause-by-clause explanation

Q238. A localisation coordinator reviews four independent translation files (French, German, Japanese and Portuguese), delegating one per turn and waiting for each reviewer to finish before starting the next. Each review takes about 40 seconds, so the full pass takes nearly three minutes. What should the coordinator do differently in its output? A) Raise each reviewer's max_tokens so it can finish in a single turn. B) Emit all four review delegations as tool_use blocks in the same assistant response, so a runtime that executes them concurrently runs them together. C) Keep one delegation per turn but shorten each reviewer's prompt so it responds faster. D) Merge the four languages into one reviewer so there is only one handoff.

Understand Q238 · the situation and each part of the question

The real situation behind this question

A software company ships each release in four languages, and translation review is the last step before launch. Product managers are waiting on a three-minute review pass several times a day during release week. None of the reviews depends on another, and the coordinator currently runs them strictly in order. The team wants to cut wait time without weakening each review.

Read the question in parts

Part 1 · four independent translation files

Independence means no review needs another's output, so they can safely overlap.

Part 2 · delegating one per turn and waiting for each reviewer to finish

The coordinator serialises independent work, which is the latency cause.

What you are being asked to decide

How does a coordinator make independent subagents run at the same time? Tests emitting multiple delegation calls in one response for concurrency.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Parallelizationrunning the four language reviews concurrently because none depends on another
tool_use blockone delegation request; four of them in a single response can run together

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: The five workflow patterns (parallelization comes in two flavors) Parallelization tool_use block Latency

Understand Q238's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Independent work should be requested together. Emit every delegation in one response so the runtime can run them concurrently, and latency approaches the slowest single task instead of the sum.

Connect the design to the situation

Four independent translation reviews run one at a time.

Best answer: B. The four reviews do not depend on each other, yet each waits for the previous one to finish. Putting all four delegations in one response lets the runtime run them at the same time. Shorter prompts or a bigger max_tokens make each review a little faster but keep them in sequence.

What each choice would do here

A · Why this does not meet the full requirement

Raise each reviewer's max_tokens so it can finish in a single turn.

A larger output cap does not change when reviews start.

If reviewers were being truncated, a bigger budget might help, but the problem is that each one waits for the previous to finish. Four sequential reviews still add up to about three minutes.

B · Why this meets the requirement

Emit all four review delegations as tool_use blocks in the same assistant response, so a runtime that executes them concurrently runs them together.

Right: independent calls in one response can run in parallel.

With four delegations in one response, a runtime such as the Agent SDK can run the reviewers at the same time. Wall-clock time drops toward the slowest single review, around 40 seconds, instead of the sum.

C · Why this does not meet the full requirement

Keep one delegation per turn but shorten each reviewer's prompt so it responds faster.

Shorter prompts trim seconds; the reviews still run one after another.

Even if each review drops to 30 seconds, four in sequence still take two minutes. The dominant cost is serial ordering of independent work, which this change does not touch.

D · Why this does not meet the full requirement

Merge the four languages into one reviewer so there is only one handoff.

One reviewer processes the languages serially inside a larger context.

A single reviewer would work through four languages in one long context, which is still sequential and risks weaker attention per language. It removes handoffs but not the serial bottleneck.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

One assistant response carrying four delegations

JSON
{
 "role": "assistant",
 "content": [
  {
   "type": "text",
   "text": "Reviewing all four locales in parallel."
  },
  {
   "type": "tool_use",
   "id": "toolu_fr",
   "name": "Agent",
   "input": {
    "description": "Review French strings",
    "prompt": "Review locales/fr.json against the glossary; return issues as a list."
   }
  },
  {
   "type": "tool_use",
   "id": "toolu_de",
   "name": "Agent",
   "input": {
    "description": "Review German strings",
    "prompt": "Review locales/de.json against the glossary; return issues as a list."
   }
  },
  {
   "type": "tool_use",
   "id": "toolu_ja",
   "name": "Agent",
   "input": {
    "description": "Review Japanese strings",
    "prompt": "Review locales/ja.json against the glossary; return issues as a list."
   }
  },
  {
   "type": "tool_use",
   "id": "toolu_pt",
   "name": "Agent",
   "input": {
    "description": "Review Portuguese strings",
    "prompt": "Review locales/pt.json against the glossary; return issues as a list."
   }
  }
 ]
}

Four tool_use blocks in one response let the runtime schedule the reviewers concurrently. The delegation tool is named Agent in current SDK docs (Task in older material). The next user message returns four tool_result blocks, one per id.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How does a coordinator make independent subagents run at the same time?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: The five workflow patterns (parallelization comes in two flavors)

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: The five workflow patterns (parallelization comes in two flavors) Parallelization tool_use block Latency

Return to Q238's situation and clause-by-clause explanation

Q239. A team asks an agent, in one instruction, to internationalise a 25-screen app: find every hard-coded string, replace each with a translation key, and reconcile duplicate keys across screens. The result is inconsistent: some screens are clean, others keep hard-coded text, and the same phrase gets different keys on different screens. Which explanation best accounts for this? A) The prompt lacked examples; two well-chosen examples in the single instruction would make every screen consistent. B) The response was truncated, so a larger max_tokens would have let the agent finish consistently. C) The model cannot edit more than a few files per request, so the work should be split only by file count. D) One instruction asked for three different jobs across 25 screens at once, so attention per job thinned; each stage needs its own focused pass.

Understand Q239 · the situation and each part of the question

The real situation behind this question

A startup is preparing to launch in three new markets and must internationalise its app first. The team hoped one instruction would do the whole job in an afternoon. Instead, translators received a key file with duplicate and conflicting keys, and several screens still show English text. The lead wants to understand why the single pass failed before trying again.

Read the question in parts

Part 1 · in one instruction

All stages were combined, which is the design choice under examination.

Part 2 · the same phrase gets different keys on different screens

Cross-screen conflicts need a global reconciliation step that the single pass never gave focus to.

What you are being asked to decide

Why does one big multi-stage instruction give inconsistent results? Tests recognising when to chain focused stages instead of one broad prompt.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Prompt chaininginventory, then per-screen replacement, then cross-screen reconciliation, each its own step
Decompositionsplitting by stage because each stage has a different job and scope

Study: A known sequence is a chain, not delegation Study: What does single-purpose mean? Prompt chaining Decomposition Workflow

Understand Q239's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When work has distinct stages, give each stage its own focused pass. Chain inventory, per-screen replacement and a global reconciliation step rather than asking one prompt to do everything at once.

Connect the design to the situation

An app's string extraction, key replacement and dedupe done in one pass come out inconsistent.

Best answer: D. Three different jobs across 25 screens in one instruction thin out attention, so some screens come out clean and others do not. Duplicate keys can only be reconciled by a pass that sees every screen. Splitting the work into its natural stages gives each job a focused pass; examples or more tokens do not.

What each choice would do here

A · Why this does not meet the full requirement

The prompt lacked examples; two well-chosen examples in the single instruction would make every screen consistent.

Examples improve format but do not split three jobs into focused passes.

Examples might make individual replacements neater, yet the agent is still juggling discovery, editing and cross-screen deduplication together. The duplicate-key problem, which needs a global view, would remain.

B · Why this does not meet the full requirement

The response was truncated, so a larger max_tokens would have let the agent finish consistently.

Truncation would cut off the end, not produce scattered inconsistency.

If output ran out, later screens would be missing entirely. Here screens are present but handled inconsistently and keys conflict, which points to divided attention rather than a length limit.

C · Why this does not meet the full requirement

The model cannot edit more than a few files per request, so the work should be split only by file count.

There is no such fixed file limit, and splitting by count ignores the stages.

Batching screens might reduce load, but each batch would still mix discovery, replacement and reconciliation, and reconciliation across batches would never happen. The natural split is by stage, not by count.

D · Why this meets the requirement

One instruction asked for three different jobs across 25 screens at once, so attention per job thinned; each stage needs its own focused pass.

Right: a chain of single-purpose stages matches the work's natural structure.

Inventory first, then per-screen replacement, then a cross-screen reconciliation pass gives each job full attention. Duplicate keys get resolved in the stage designed for exactly that, instead of being an afterthought mid-edit.

Check your understanding · explain the design in your own words

Why does one big multi-stage instruction give inconsistent results?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: A known sequence is a chain, not delegation Study: What does single-purpose mean?

Study: A known sequence is a chain, not delegation Study: What does single-purpose mean? Prompt chaining Decomposition Workflow

Return to Q239's situation and clause-by-clause explanation

Q240. A product analyst spent an hour in an agent session building a churn analysis of subscription cohorts. She now wants to test two pricing ideas, an annual-plan discount and a usage-based tier, each starting from that analysis, without either exploration influencing the other. Which statement about using a session fork for this is correct? A) A fork keeps only a compacted summary of the original session, trading detail for isolation. B) A fork is equivalent to resuming; both simply append new turns to the original session's history. C) A fork creates a new session whose history begins as a copy of the original at the fork point, so each branch keeps the full analysis and neither sees the other's later turns. D) A fork shares one live history, so turns in either branch appear in both and keep them in sync.

Understand Q240 · the situation and each part of the question

The real situation behind this question

A subscription company is losing customers, and the pricing committee meets next week. The analyst's churn analysis took an hour of queries and reasoning. She must present two pricing options, and the committee needs to trust that each was judged on its own merits, not shaped by conclusions drawn while exploring the other. She is deciding how to structure the sessions.

Read the question in parts

Part 1 · each starting from that analysis

Both explorations need the full baseline, so a summary would lose detail.

Part 2 · without either exploration influencing the other

After the branch point the histories must be separate, which rules out resuming one session twice.

What you are being asked to decide

What exactly does forking a session give you? Tests the precise semantics of a session fork versus resume.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Checkpointthe finished churn analysis, where both branches begin
Agent SDKthe runtime that forks a saved session when resuming with the fork option

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names Checkpoint Context isolation Agent SDK

Understand Q240's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A fork gives each branch the same starting history and then keeps them apart. Use it when alternatives must share a rich baseline but not each other's reasoning; resuming would mix them, and a summary would lose detail.

Connect the design to the situation

Two pricing ideas must each start from one hour-long churn analysis.

Best answer: C. The analyst needs both explorations to start from the same full analysis and then stay apart. A fork copies the history available at the fork point into a new session, and the branches diverge from there. Resuming appends to one shared history, and a summary drops detail.

What each choice would do here

A · Why this does not meet the full requirement

A fork keeps only a compacted summary of the original session, trading detail for isolation.

A fork copies the available history rather than summarising it.

The branch receives the runtime's saved conversation history, not a rewritten summary. Details already lost to earlier compaction are not restored, but the fork itself does not summarise.

B · Why this does not meet the full requirement

A fork is equivalent to resuming; both simply append new turns to the original session's history.

Resume continues the same session; fork creates a new one.

Resuming twice in sequence would put both explorations into one history, so the second would see the first. Forking gives each exploration its own session identity and history.

C · Why this meets the requirement

A fork creates a new session whose history begins as a copy of the original at the fork point, so each branch keeps the full analysis and neither sees the other's later turns.

Right: shared starting history, independent futures.

Each pricing exploration starts with the complete churn analysis, including tool results and intermediate reasoning. What the discount branch concludes never appears in the usage-tier branch, and the original session is left unchanged.

D · Why this does not meet the full requirement

A fork shares one live history, so turns in either branch appear in both and keep them in sync.

That would defeat the purpose; branches are independent after the fork.

If turns were shared, the discount exploration would contaminate the usage-tier exploration, which is exactly what the analyst wants to avoid. Forks diverge from the fork point onward.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Forking from a saved baseline in the Python Agent SDK

PYTHON
from claude_agent_sdk import query, ClaudeAgentOptions

async def explore(baseline_id, idea):
    opts = ClaudeAgentOptions(resume=baseline_id, fork_session=True)
    async for msg in query(prompt=f"Using the churn analysis so far, evaluate: {idea}", options=opts):
        print(msg)

# await explore(saved_id, 'annual-plan discount')
# await explore(saved_id, 'usage-based tier')

resume points at the saved analysis session and fork_session=True makes each call start a new session copied from it. The two explorations never write into the baseline or into each other.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What exactly does forking a session give you?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names

Study: Fork, resume, or start fresh? Study: Sessions, forking, and spawning — the mechanics the exam names Checkpoint Context isolation Agent SDK

Return to Q240's situation and clause-by-clause explanation

Q241. A pre-launch readiness review spawns legal, accessibility and performance reviewers in parallel. All three return successfully and no errors are logged, yet the final readiness report contains only accessibility findings. Which control would have caught this before the report went out? A) Before synthesis, the coordinator checks that it holds a result for every reviewer it dispatched and passes all of them into the synthesis step. B) Tell the synthesis step to write "no issues found" for any area without findings. C) Automatically retry any reviewer whose call returns an error. D) Raise each reviewer's turn limit so it has room to finish its analysis.

Understand Q241 · the situation and each part of the question

The real situation behind this question

A product team relies on the readiness report to decide whether a release ships. A report that silently omits legal and performance findings could put a non-compliant or slow product in front of customers. Everything looked healthy: no errors, three successful reviewers. The engineering manager wants a control that catches this class of silent loss before a release decision is made.

Read the question in parts

Part 1 · All three return successfully and no errors are logged

Reviewer failure is ruled out, so the loss happened after results came back.

Part 2 · before the report went out

The control must act before synthesis output is used, not diagnose afterward.

What you are being asked to decide

What check stops successful subagent results from being dropped before synthesis? Tests that the coordinator must gather all expected results before synthesising.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Orchestratorthe coordinator responsible for collecting all three reviews
Handoffthe synthesis input, which must include every returned review

Study: Successful workers can still produce an incomplete report Study: Follow the information through six checkpoints Orchestrator Handoff Hub-and-spoke

Understand Q241's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Successful subagents are useless if their results never reach synthesis. The coordinator should treat every dispatched task as a required input, confirm all results are present, and pass them all forward.

Connect the design to the situation

Three reviewers succeed but only one appears in the final report.

Best answer: A. Every reviewer succeeded, so the loss happened after they returned, when the coordinator built the synthesis input. A completeness check against what was dispatched catches that. Retries and turn limits address failures that did not happen, and defaulting to 'no issues' turns missing data into a false all-clear.

What each choice would do here

A · Why this meets the requirement

Before synthesis, the coordinator checks that it holds a result for every reviewer it dispatched and passes all of them into the synthesis step.

Right: a completeness check on expected inputs catches the dropped results.

The coordinator knows it dispatched three reviewers. Checking for three results, and feeding all three to synthesis, would reveal that legal and performance findings were never forwarded, before anyone relied on the report.

B · Why this does not meet the full requirement

Tell the synthesis step to write "no issues found" for any area without findings.

This would hide the problem by presenting missing data as a clean result.

Legal and performance would be reported as issue-free when their findings were simply never passed along. That turns a visible gap into a false assurance, which is worse than the original failure.

C · Why this does not meet the full requirement

Automatically retry any reviewer whose call returns an error.

No reviewer failed, so a retry rule would never fire.

All three reviewers succeeded. Error-driven retries are a sound resilience control, but they cannot detect results that exist yet were never passed on to synthesis.

D · Why this does not meet the full requirement

Raise each reviewer's turn limit so it has room to finish its analysis.

The reviewers already finished; turn limits are not the issue.

Higher limits help when work is cut off, which would show up as incomplete or failed results. Here every reviewer completed successfully, and the findings were lost after they returned.

Check your understanding · explain the design in your own words

What check stops successful subagent results from being dropped before synthesis?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Successful workers can still produce an incomplete report Study: Follow the information through six checkpoints

Study: Successful workers can still produce an incomplete report Study: Follow the information through six checkpoints Orchestrator Handoff Hub-and-spoke

Return to Q241's situation and clause-by-clause explanation

Q242. A compliance agent scans 60 vendor contracts for a nonstandard indemnity clause and runs with a 10-turn limit. In about half the runs it stops after roughly 35 contracts, missing later matches; in the rest it reaches all 60. No tool errors are logged. What is the root cause? A) Contracts after the 35th are longer, so the model stops reading them in detail. B) The system prompt does not list all 60 contract file names, so the model loses track of which contracts remain. C) The turn limit is set below what the task needs, so the loop is cut off mid-scan; completion should come from end_turn, with any cap kept as a generous safety ceiling. D) A rate limit silently drops tool calls after about 35 contracts.

Understand Q242 · the situation and each part of the question

The real situation behind this question

A procurement team must find every vendor contract with a risky indemnity clause before renewals. Missing even one could expose the company to uncapped liability. The agent's results are inconsistent: sometimes complete, sometimes stopping two-thirds of the way through, always without errors. The compliance lead needs to know why before trusting any result, because the renewal deadline is weeks away and legal cannot review all 60 contracts by hand.

Read the question in parts

Part 1 · runs with a 10-turn limit

The loop has a fixed turn budget, which is the prime suspect for clean early stops.

Part 2 · No tool errors are logged

Nothing failed, so the stop came from the loop's own control logic rather than a fault.

What you are being asked to decide

Why does an agent sometimes stop halfway through a scan with no errors? Tests recognising a turn cap that truncates work, and using end_turn as the completion signal.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Loop token budgetthe 10-turn limit, sized below what some runs need
stop_reasonend_turn is the signal that the agent has actually finished

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object Loop token budget stop_reason Agent loop

Understand Q242's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When runs stop cleanly partway through with no errors, suspect the loop's own limit. Let end_turn signal that the work is finished, and keep a turn ceiling only as a generous safety net.

Connect the design to the situation

A contract scan stops at about 35 of 60 files in half the runs.

Best answer: C. Half the runs stop mid-scan with no errors, which is what a loop does when it runs out of turns. A file list, rate limits or document length would not stop the run cleanly at the same point. Let end_turn signal completion and keep the cap only as a generous safety ceiling.

What each choice would do here

A · Why this does not meet the full requirement

Contracts after the 35th are longer, so the model stops reading them in detail.

Document length would cause uneven analysis, not a clean stop with no errors.

If later contracts were harder, the agent would still read them, perhaps less carefully. Here it simply stops, which is what a loop does when its turn budget runs out.

B · Why this does not meet the full requirement

The system prompt does not list all 60 contract file names, so the model loses track of which contracts remain.

A list might help planning but cannot give the loop more turns.

Even with every file named, a run that needs twelve turns still stops at ten. The split between runs that finish and runs that stop matches turn usage, not awareness of the file list.

C · Why this meets the requirement

The turn limit is set below what the task needs, so the loop is cut off mid-scan; completion should come from end_turn, with any cap kept as a generous safety ceiling.

Right: the cap, not the model, is ending the search early.

Runs that batch reads efficiently fit in ten turns, while runs that read fewer contracts per turn hit the limit around 35. Letting end_turn signal completion, with a much higher ceiling for safety, lets every run finish.

D · Why this does not meet the full requirement

A rate limit silently drops tool calls after about 35 contracts.

Rate limits produce errors or retries, and none are logged.

Dropped calls would appear as errors or failed tool results. The clean logs and the bimodal pattern, finishing or stopping near the same point, point to a turn budget rather than throttling.

Check your understanding · explain the design in your own words

Why does an agent sometimes stop halfway through a scan with no errors?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object

Study: Stop reasons — memorize this table Study: The agentic loop — the exam's core object Loop token budget stop_reason Agent loop

Return to Q242's situation and clause-by-clause explanation

Q243. A team configures an incident-postmortem orchestrator with the Agent SDK. Its system prompt says to hand timeline reconstruction and log analysis to the subagents defined in its configuration. Every run finishes in a single turn with the orchestrator writing a shallow postmortem itself; no subagent ever starts and no errors appear. Which configuration problem is MOST likely? A) The permission mode is set to accept edits, which disables all subagents. B) The delegation tool (named Agent in current SDK docs, Task in older material) is not among the tools available to the orchestrator, so it has nothing to call. C) The subagent definitions omit a model field, so they cannot start. D) The system prompt says "hand off" instead of naming each subagent's identifier.

Understand Q243 · the situation and each part of the question

The real situation behind this question

An SRE team wants detailed postmortems within hours of each incident, so it built an orchestrator that farms out timeline reconstruction and log analysis. Instead, it gets thin reports written in one pass. The configuration looks right at a glance, and nothing errors. The team needs to find which setting keeps the orchestrator from ever delegating, since the subagent definitions themselves appear complete and correct.

Read the question in parts

Part 1 · Every run finishes in a single turn

The orchestrator never attempts a tool call, so it likely has no delegation tool to call.

Part 2 · no errors appear

Nothing is failing, which rules out broken subagent definitions and points to a missing capability.

What you are being asked to decide

Why does an orchestrator told to delegate never start a subagent? Tests that delegation requires the delegation tool to be available, not just instructed.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Tool availability and permissionswhether the delegation tool is on the orchestrator's menu at all
Agent SDKthe runtime where subagents are defined and started through a delegation tool

Study: The Task tool — how subagents are actually born Tool availability and permissions Allowlist Agent SDK

Understand Q243's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Instructions cannot grant tools. If the orchestrator should delegate, the delegation tool must be available to it; otherwise it will do the work itself, silently.

Connect the design to the situation

A postmortem orchestrator always writes the report itself in one turn.

Best answer: B. The orchestrator never tries to delegate and nothing errors, which means it has no delegation tool to call. Prompt wording and optional subagent fields cannot cause that. The delegation tool must be among the tools the orchestrator actually has.

What each choice would do here

A · Why this does not meet the full requirement

The permission mode is set to accept edits, which disables all subagents.

Permission modes govern approvals for actions; they do not remove subagents.

Accept-edits mode changes which file edits need approval. It does not hide the delegation tool or prevent subagents from starting, so it cannot explain a coordinator that never tries to delegate.

B · Why this meets the requirement

The delegation tool (named Agent in current SDK docs, Task in older material) is not among the tools available to the orchestrator, so it has nothing to call.

Right: without the delegation tool on its menu, the orchestrator can only answer directly.

The prompt describes delegation, but the model can only call tools it is given. Seeing no delegation tool, it does the reasonable thing and writes the postmortem itself in one turn, with no error because nothing failed.

C · Why this does not meet the full requirement

The subagent definitions omit a model field, so they cannot start.

The model field is optional; subagents fall back to a default.

A missing model setting does not stop a subagent from being spawned. And if spawning were attempted and failed, there would be an error or a failed tool result, not a clean single-turn answer.

D · Why this does not meet the full requirement

The system prompt says "hand off" instead of naming each subagent's identifier.

Wording affects which subagent is chosen, not whether delegation is possible.

Naming subagents precisely can improve selection once delegation works. It cannot explain why no delegation is ever attempted, since the model would still try to delegate if the tool were available.

Check your understanding · explain the design in your own words

Why does an orchestrator told to delegate never start a subagent?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The Task tool — how subagents are actually born

Study: The Task tool — how subagents are actually born Tool availability and permissions Allowlist Agent SDK

Return to Q243's situation and clause-by-clause explanation

Q244. An inventory-reconciliation agent's loop returns as soon as a response contains any text block. When the model writes "Let me check the warehouse count" and also requests a tool in the same response, the loop exits before the tool ever runs. Which rule should the loop follow instead? A) Return when the text mentions a final reconciled count; otherwise keep looping. B) If stop_reason is tool_use, run every requested tool, append the tool_result blocks in a user message and call again; if it is end_turn, return the answer. C) Return when the response contains any text; otherwise execute the tools. D) Run exactly five iterations and return whatever text the last response contains.

Understand Q244 · the situation and each part of the question

The real situation behind this question

A retailer's reconciliation agent compares system inventory to warehouse counts overnight. Staff found that many runs returned a promise to check the warehouse instead of an actual reconciliation, so discrepancies went unflagged for days. The developer traced it to the loop's exit rule and now needs the rule that handles responses mixing narration and tool calls, so every run ends only when the work is truly finished.

Read the question in parts

Part 1 · returns as soon as a response contains any text block

The current exit rule treats narration as completion, which is the defect.

Part 2 · also requests a tool in the same response

Responses can mix text and tool requests, so text alone cannot mean done.

What you are being asked to decide

What rule should decide whether an agent loop continues or returns? Tests the exact stop_reason branching for an agent loop.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
stop_reasontool_use means run the requested tools; end_turn means return
tool_resultthe block that carries the warehouse lookup back to the model

Study: Stop reasons — memorize this table Study: The messages array — how the loop physically grows stop_reason tool_result Agent loop

Understand Q244's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A response's stop_reason, not its content, says whether the model is done. Run every requested tool and loop on tool_use; return on end_turn; handle other reasons explicitly.

Connect the design to the situation

A loop quits when the model narrates before calling a tool.

Best answer: B. Text and tool requests often arrive in the same response, so text is not a completion signal. The loop should branch on stop_reason: run the tools and continue on tool_use, stop on end_turn. Matching phrases or fixing an iteration count both guess at completion.

What each choice would do here

A · Why this does not meet the full requirement

Return when the text mentions a final reconciled count; otherwise keep looping.

Parsing prose for completion is fragile and can loop forever or stop early.

The model may phrase the final count in many ways or mention interim counts along the way. Completion becomes a guess about wording instead of the structured signal the API provides.

B · Why this meets the requirement

If stop_reason is tool_use, run every requested tool, append the tool_result blocks in a user message and call again; if it is end_turn, return the answer.

Right: the stop reason, not the presence of text, decides whether to continue.

A response can contain both text and tool requests. Branching on stop_reason runs the warehouse lookup, feeds back the result, and only returns once the model signals it is done.

C · Why this does not meet the full requirement

Return when the response contains any text; otherwise execute the tools.

This is the current bug: text and tool requests often arrive together.

Models regularly narrate before calling tools. Treating any text as final ends the loop exactly when work is about to begin, which is the failure the team is seeing.

D · Why this does not meet the full requirement

Run exactly five iterations and return whatever text the last response contains.

A fixed count ignores whether the work is finished.

Simple reconciliations would waste turns and complex ones would stop mid-task, returning partial text as if final. A cap is a safety limit, not a completion rule.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

A loop that branches on stop_reason

PYTHON
def run(client, model, messages, tools, execute):
    while True:
        resp = client.messages.create(model=model, max_tokens=2048, tools=tools, messages=messages)
        messages.append({"role": "assistant", "content": resp.content})
        if resp.stop_reason == "tool_use":
            results = [
                {"type": "tool_result", "tool_use_id": b.id, "content": execute(b.name, b.input)}
                for b in resp.content if b.type == "tool_use"
            ]
            messages.append({"role": "user", "content": results})
            continue
        if resp.stop_reason == "end_turn":
            return resp
        raise RuntimeError(f"unhandled stop_reason: {resp.stop_reason}")
🏭 Production shape: the SDK tool runner owns the loop

In production the SDK tool runner does this loop for you. It sends each tool_result back with the matching id and stops when the model ends its turn. You still check the final stop_reason, because a stop for max_tokens, refusal or the iteration cap is not success.

PYTHON
import os
from typing import Annotated

import anthropic
from anthropic import beta_tool
from pydantic import Field

from app.repo import code_search  # your own module

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-5")


@beta_tool(
    name="search_code",
    description="Search the repository and return matching paths with line numbers.",
)
def search_code(pattern: Annotated[str, Field(description="Text or a regular expression to find, e.g. legacyAuthenticate.")]) -> str:
    return code_search.grep(pattern)


def run(task: str):
    runner = client.beta.messages.tool_runner(
        model=MODEL, max_tokens=2048, tools=[search_code],
        messages=[{"role": "user", "content": task}],
        max_iterations=20,  # a safety ceiling, not the success signal
    )
    last = None
    for last in runner:  # one assistant message per turn; narration text does not stop it
        print(last.stop_reason, last.usage.output_tokens)
    if last is not None and last.stop_reason == "end_turn":
        return last
    raise RuntimeError(f"stopped without end_turn: {last and last.stop_reason}")

Tools in app/tools/repo.py; the runner call in app/agent.py.

The narration text is kept in history but does not end the loop. Every tool_use block gets a matching tool_result in the next user message, and only end_turn returns. Other reasons such as max_tokens are surfaced rather than treated as success.

Not supplied in this example · settings and defaults

context_management — not supplied here. No server compaction strategy is enabled by this body. Host/manual compaction is separate.

output_config — not supplied here. No output_config schema or effort override is supplied. A prompt asking for JSON does not provide schema enforcement.

tool_choice — not supplied here. With tools supplied, the default is auto: the model chooses whether to request a tool. Without tools, no supplied tool can be selected.

system — not supplied here. No application system instructions are supplied through this field. User messages still provide task text.

stop_sequences — not supplied here. No custom stop strings are configured. Generation can still stop for end_turn, tool_use, max_tokens, or another documented reason.

stream — not supplied here. For a direct non-streaming create/parse call, the result is one message rather than an event stream. A stream helper configures this separately.

thinking — not supplied here. No explicit thinking override is shown. Behavior depends on the selected model; omission does not universally mean thinking is off.

cache_control — not supplied here. No top-level automatic cache marker is supplied. Check nested system/tool/content blocks for separate cache_control markers.

metadata — not supplied here. No application user reference is supplied through metadata. Authentication still comes from the request credentials.

betas — not explicitly supplied. No beta list appears in these calls. SDK/client configuration can add headers; inspect those before assuming no beta header is sent.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What rule should decide whether an agent loop continues or returns?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Stop reasons — memorize this table Study: The messages array — how the loop physically grows

Study: Stop reasons — memorize this table Study: The messages array — how the loop physically grows stop_reason tool_result Agent loop

Return to Q244's situation and clause-by-clause explanation

Q245. A property-valuation assistant asks a comparable-sales subagent (about 2.4 s) and then a zoning-rules subagent (about 2.1 s), one after the other, for each address. End-to-end latency is about 5 s, and the two lookups are independent. Which change lowers latency the most, and to roughly what? A) Request both subagents in the same coordinator response so they overlap, bringing latency down toward the slower lookup, around 2.5 to 3 s. B) Put a 1-second timeout on the zoning subagent so the sales subagent can start sooner. C) Switch the coordinator to a smaller, faster model, because its planning turns dominate the 5 seconds. D) Fold both lookups into the coordinator's own context to avoid the cost of spawning subagents.

Understand Q245 · the situation and each part of the question

The real situation behind this question

An estate agency's valuation assistant is used live during client calls, and agents feel a five-second pause is too long. The two lookups it runs share no inputs. The engineering team wants the biggest latency reduction without weakening either lookup, and needs to know both what to change and what latency to expect afterward so it can report back to sales.

Read the question in parts

Part 1 · one after the other

The lookups are serialised, which makes total latency the sum of both.

Part 2 · the two lookups are independent

Independence means they can safely run at the same time.

What you are being asked to decide

How much faster can two independent lookups get, and how? Tests concurrent delegation of independent subtasks and its effect on latency.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Parallelizationrunning both lookups at once because neither needs the other's result
Latencytime per valuation, which falls toward the slower lookup when they overlap

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: What overlaps and what must wait? Parallelization Latency Throughput

Understand Q245's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Independent tasks should run together. Request both lookups in one coordinator response and latency drops toward the slower lookup instead of the sum; tuning the coordinator or truncating a lookup barely moves it.

Connect the design to the situation

Comparable sales and zoning lookups run back to back for every valuation.

Best answer: A. The two lookups do not depend on each other but run back to back, so latency is roughly their sum. Requesting both at once lets them overlap, bringing latency close to the slower one. The coordinator's planning time is a small share of the five seconds.

What each choice would do here

A · Why this meets the requirement

Request both subagents in the same coordinator response so they overlap, bringing latency down toward the slower lookup, around 2.5 to 3 s.

Right: concurrent independent work costs about the longest task plus coordination.

With both delegations issued together, the zoning lookup runs while comparable sales is still working. Total time approaches the 2.4 s lookup plus coordinator overhead, instead of stacking both.

B · Why this does not meet the full requirement

Put a 1-second timeout on the zoning subagent so the sales subagent can start sooner.

A timeout truncates results; it does not overlap the work.

Cutting zoning off at one second would often return nothing useful, degrading the valuation. And because the calls are still sequential, the saving comes from lost work, not from concurrency.

C · Why this does not meet the full requirement

Switch the coordinator to a smaller, faster model, because its planning turns dominate the 5 seconds.

The numbers show the subagents, not planning, make up most of the time.

The two lookups account for about 4.5 of the 5 seconds. A faster coordinator might shave a fraction of the remaining half-second, but the serial lookups still dominate.

D · Why this does not meet the full requirement

Fold both lookups into the coordinator's own context to avoid the cost of spawning subagents.

Doing both lookups inline keeps them sequential and enlarges the coordinator's context.

Spawn overhead is small compared with the lookups themselves. Running both inside the coordinator still does one after the other, so latency stays near the sum.

Check your understanding · explain the design in your own words

How much faster can two independent lookups get, and how?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: What overlaps and what must wait?

Study: Two axes, two owners — control flow vs throughput (the trap that costs points) Study: What overlaps and what must wait? Parallelization Latency Throughput

Return to Q245's situation and clause-by-clause explanation

Q246. A curriculum builder uses research subagents to gather material, then drafts a course outline in a single pass. On broad subjects such as "climate science for policymakers", some modules end up thin or empty, but the coordinator declares the outline finished anyway. Which coordination pattern most directly improves coverage? A) Have the final outline list thin modules in an appendix so readers know where to look elsewhere. B) Give each research subagent a larger output budget so it returns more material per call. C) Spawn twice as many research subagents at the start of every run. D) Have the coordinator check the draft against the intended coverage, send targeted research tasks for thin modules, and repeat until every module meets the bar.

Understand Q246 · the situation and each part of the question

The real situation behind this question

An education company sells short courses to government staff. Broad subjects are its best sellers, but customers have complained that some modules are nearly empty. The current coordinator gathers research once and writes the outline, trusting whatever came back. The content lead wants a change that reliably fills weak modules, without making every course run cost twice as much, including the narrow ones.

Read the question in parts

Part 1 · in a single pass

There is no evaluation step, so the coordinator never checks what came back.

Part 2 · declares the outline finished anyway

Completion is declared without comparing the result to the intended coverage.

What you are being asked to decide

What pattern makes a coordinator fix thin coverage instead of declaring done? Tests adding an evaluate-and-redelegate loop to a single-pass coordinator.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Evaluator-optimizerthe loop that checks the outline and sends work back for the weak parts
Orchestratorthe coordinator that must own the check before declaring done

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report Evaluator-optimizer Orchestrator Decomposition

Understand Q246's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A single pass cannot see its own gaps. Add an evaluation step: compare the draft with the intended coverage, send targeted work for what is thin, and stop only when the bar is met.

Connect the design to the situation

A course outline on a broad subject has thin modules that nobody revisits.

Best answer: D. A single pass cannot notice its own gaps. Adding an evaluation step that compares the draft with the intended coverage, then sending targeted research for the thin modules, closes gaps where they are. More workers or bigger outputs spend effort without knowing what is missing.

What each choice would do here

A · Why this does not meet the full requirement

Have the final outline list thin modules in an appendix so readers know where to look elsewhere.

Flagging gaps is transparent but leaves the outline incomplete.

Readers learn which modules are weak, yet the system had the ability to fill them and chose not to. This is a disclosure, not a coverage improvement.

B · Why this does not meet the full requirement

Give each research subagent a larger output budget so it returns more material per call.

More material per call does not target the modules that are missing.

Workers will return longer results on the topics they were given, which may already be well covered. Thin modules stay thin unless someone notices them and asks for them specifically.

C · Why this does not meet the full requirement

Spawn twice as many research subagents at the start of every run.

More upfront research is spent blindly and still cannot notice what is missing.

Doubling the workers increases cost on every run, including narrow subjects that were fine. Without an evaluation step, the coordinator still has no way to see which modules came back thin.

D · Why this meets the requirement

Have the coordinator check the draft against the intended coverage, send targeted research tasks for thin modules, and repeat until every module meets the bar.

Right: an evaluate-and-redelegate loop finds and fills gaps.

The coordinator compares the outline with the planned module list, sees that, say, adaptation finance is thin, and sends a focused research task for exactly that. It repeats until coverage is sufficient, rather than trusting one pass.

Check your understanding · explain the design in your own words

What pattern makes a coordinator fix thin coverage instead of declaring done?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report

Study: Evaluator-optimizer: follow one draft Study: Coordinator communication from plan to final report Evaluator-optimizer Orchestrator Decomposition

Return to Q246's situation and clause-by-clause explanation

Q247. A meal-planning service runs a nutrition subagent and an allergy subagent, then spawns a plan-writer subagent. Both analysis subagents succeed, but the meal plans reflect only the nutrition findings, and some include foods the allergy check flagged. Logs show the plan-writer's prompt contained the nutrition output alone. Which statement is exactly right? A) Subagents automatically inherit the coordinator's full history, so the allergy output must have been truncated by the plan writer. B) The plan writer knows only what its delegation prompt contains or explicitly references, so the allergy findings never reached it. C) The allergy results were returned as structured JSON, which subagent prompts cannot accept. D) Subagents can read each other's results through a shared session store unless isolation is turned off.

Understand Q247 · the situation and each part of the question

The real situation behind this question

A meal-planning service promises users plans that respect their allergies. Some users received plans containing foods their allergy check had already flagged, a real safety risk. Every subagent reported success, which confused the team. The logs point at the plan writer's input, and the team needs a precise understanding of what subagents can see before redesigning the handoffs between the analysis steps and the writer.

Read the question in parts

Part 1 · Logs show the plan-writer's prompt contained the nutrition output alone

The evidence locates the gap in what the coordinator passed, not in what the writer did.

Part 2 · Both analysis subagents succeed

The allergy findings existed, so the loss happened in the handoff.

What you are being asked to decide

What does a subagent actually know when it starts? Tests that subagents receive only what the coordinator passes in.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Context isolationthe plan writer sees its own brief, not its siblings' results
Handoffthe plan writer's prompt, which must carry both nutrition and allergy findings

Study: Coordinator handoffs: wait, pass context, then continue Study: Where each instruction and piece of data goes Context isolation Handoff Subagent

Understand Q247's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Subagents start from their brief. If a later step needs earlier findings, the coordinator must include them or point to them explicitly; here, the allergy results belong in the plan writer's prompt.

Connect the design to the situation

Meal plans ignore allergy findings because the writer never received them.

Best answer: B. The logs show the plan writer's prompt held only the nutrition output. A subagent does not automatically see its siblings' results or the coordinator's history, so the allergy findings never reached it. The coordinator has to put every relevant result into the plan writer's brief.

What each choice would do here

A · Why this does not meet the full requirement

Subagents automatically inherit the coordinator's full history, so the allergy output must have been truncated by the plan writer.

A subagent does not automatically receive the coordinator's whole conversation.

The logs show the prompt itself lacked the allergy output, so nothing was truncated downstream. Assuming automatic inheritance hides the real gap in the coordinator's handoff.

B · Why this meets the requirement

The plan writer knows only what its delegation prompt contains or explicitly references, so the allergy findings never reached it.

Right: sibling results must be forwarded by the coordinator.

The plan writer did its job with the information it had. The allergy results existed but were never included, so the coordinator must add them to the plan writer's brief alongside the nutrition output.

C · Why this does not meet the full requirement

The allergy results were returned as structured JSON, which subagent prompts cannot accept.

Structured results can be placed in a prompt like any other text.

JSON is a normal way to pass findings between steps. The problem is that the coordinator left the allergy output out, not that its format was incompatible.

D · Why this does not meet the full requirement

Subagents can read each other's results through a shared session store unless isolation is turned off.

There is no automatic shared store of sibling results to switch on.

Workers see what they are given. Relying on an imagined shared store would leave the handoff missing and the plans unsafe for users with allergies.

Check your understanding · explain the design in your own words

What does a subagent actually know when it starts?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Coordinator handoffs: wait, pass context, then continue Study: Where each instruction and piece of data goes

Study: Coordinator handoffs: wait, pass context, then continue Study: Where each instruction and piece of data goes Context isolation Handoff Subagent

Return to Q247's situation and clause-by-clause explanation

Q248. A policy institute asks its research system about the consequences of a four-day work week. The coordinator's plan lists four subtasks, all about employee outcomes (productivity, burnout, retention, commuting), and the report never covers employer costs, customer-service coverage or effects on public services. Which change prevents this class of failure? A) Have the coordinator check its subtask list against the full scope of the question, adding missing dimensions before any delegation. B) Instruct search workers to broaden their queries beyond the subtopic they were assigned. C) Tell the synthesis worker to add sections for obvious omissions from its own general knowledge. D) Loosen the analysis worker's relevance filter so more borderline sources pass through.

Understand Q248 · the situation and each part of the question

The real situation behind this question

A policy institute is briefing lawmakers on a four-day work-week bill. A report that covers only employee outcomes could mislead them about costs to businesses and public services. The research workers did good work within their topics, so the institute wants a change that stops the same narrow-planning failure from recurring on future questions, not just a patch for this one.

Read the question in parts

Part 1 · all about employee outcomes

The plan itself is one-sided, which locates the failure at decomposition.

Part 2 · prevents this class of failure

The answer must be preventive and general, acting before workers are dispatched.

What you are being asked to decide

How do you stop a research coordinator from planning too narrowly? Tests preventive coverage checks on the coordinator's decomposition.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Decompositionthe four employee-only subtasks, which bounded what the report could cover
Orchestratorthe coordinator that must check its plan for missing stakeholders before delegating

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure Decomposition Orchestrator Provenance

Understand Q248's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The report can only be as broad as the plan. Make the coordinator check its subtask list against every dimension and stakeholder the question implies before delegating, so gaps are closed at the source.

Connect the design to the situation

A four-day-week report covers only employee outcomes.

Best answer: A. Every subtask was about employees, so the missing dimensions were never assigned. The fix is at planning time: check the subtask list against the full scope of the question before delegating. Wider queries, looser filters or a synthesis step writing from memory cannot add topics nobody was sent to research.

What each choice would do here

A · Why this meets the requirement

Have the coordinator check its subtask list against the full scope of the question, adding missing dimensions before any delegation.

Right: the gap begins in the plan, so the plan must be checked for coverage.

Before delegating, the coordinator asks which stakeholders and dimensions the question implies: employees, employers, customers, public services. Adding the missing ones up front means workers research everything that matters.

B · Why this does not meet the full requirement

Instruct search workers to broaden their queries beyond the subtopic they were assigned.

Scope creep in workers produces overlap and noise, not reliable coverage.

Workers straying beyond their assignment might stumble on employer costs, but coverage becomes accidental and duplicated. The plan still does not say that those dimensions matter.

C · Why this does not meet the full requirement

Tell the synthesis worker to add sections for obvious omissions from its own general knowledge.

Synthesis would fill gaps with unsourced content.

The report would gain sections on employer costs and public services, but without research behind them. For a policy institute, unsourced claims are a credibility risk, not a fix.

D · Why this does not meet the full requirement

Loosen the analysis worker's relevance filter so more borderline sources pass through.

Filters operate within assigned topics; they cannot add unassigned ones.

More permissive filtering lets extra employee-outcome sources through. It cannot produce research on customer-service coverage because no worker was ever sent to look for it.

Check your understanding · explain the design in your own words

How do you stop a research coordinator from planning too narrowly?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure

Study: Successful workers can still produce an incomplete report Study: Diagnose the earliest evidenced failure Decomposition Orchestrator Provenance

Return to Q248's situation and clause-by-clause explanation

Q249. A medical-billing pipeline runs three steps: identify the claim type, assign billing codes, then check that codes agree with the diagnosis. Dental claims reach 71% accuracy while outpatient claims reach 96%. An engineering lead proposes merging all three steps into one prompt to cut cost by more than half. What evidence would need to exist before the merge is acceptable? A) A pilot running the merged prompt on outpatient claims only, since they are already accurate. B) An evaluation on labelled claims showing the merged prompt matches the pipeline's accuracy for each claim type, dental included, and still catches code-diagnosis mismatches. C) A latency benchmark showing the merged prompt is faster on average. D) Sign-off from the product owner accepting somewhat lower accuracy on dental claims.

Understand Q249 · the situation and each part of the question

The real situation behind this question

A healthcare billing company processes thousands of claims a day, and wrong codes mean rejected claims, delayed payment and audit exposure. Finance wants costs down, and the merge looks attractive on paper. The quality team worries that dental claims, already the weakest, could get worse without the dedicated consistency step. Leadership asks what evidence would settle the debate before any change reaches production.

Read the question in parts

Part 1 · Dental claims reach 71% accuracy

Quality is already weak for one type, so that type decides whether a change is safe.

Part 2 · What evidence would need to exist

The question asks for the decisive proof, not an opinion or approval.

What you are being asked to decide

What proof is needed before merging focused pipeline steps into one prompt? Tests requiring per-type quality evidence before trading steps for speed or cost.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Accuracy-latency tradeoffthe merge saves cost and time but may cost accuracy where it is already low
Golden setlabelled claims by type, used to compare the merged prompt with the pipeline

Study: Faster pipelines: measure quality before merging steps Study: Decision rule: keep the focused stages until the candidate is measured Accuracy-latency tradeoff Eval Golden set

Understand Q249's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Focused steps stay until a merged candidate is shown to match them where it matters most. Evaluate the merge per claim type, dental first, and confirm the consistency check still works before accepting the savings.

Connect the design to the situation

A billing pipeline weak on dental claims is proposed for a cost-cutting merge.

Best answer: B. Dental claims are already the weak area, and merging would remove the focused consistency check. Only a per-type evaluation that includes dental shows whether the merged prompt holds up. A speed benchmark, a sign-off or an outpatient-only pilot all skip the risk that matters.

What each choice would do here

A · Why this does not meet the full requirement

A pilot running the merged prompt on outpatient claims only, since they are already accurate.

Evidence from the strong type says nothing about the weak one.

Outpatient success would be encouraging, but the concern is dental, where the pipeline already struggles. The pilot would avoid the exact case that needs testing before any merge.

B · Why this meets the requirement

An evaluation on labelled claims showing the merged prompt matches the pipeline's accuracy for each claim type, dental included, and still catches code-diagnosis mismatches.

Right: the merge must be measured against per-type quality, especially where it is weakest.

Dental is already the weak spot, and merging removes the focused consistency step. Only a per-type evaluation shows whether the merged prompt holds quality there; if it does, the saving is safe.

C · Why this does not meet the full requirement

A latency benchmark showing the merged prompt is faster on average.

Speed is already expected; the risk is accuracy.

Everyone agrees the merge will be faster and cheaper. An average speed benchmark says nothing about whether dental accuracy falls further or whether mismatches slip past.

D · Why this does not meet the full requirement

Sign-off from the product owner accepting somewhat lower accuracy on dental claims.

Accepting a loss without measuring it is not evidence.

A sign-off shifts responsibility but does not tell anyone how much accuracy would drop. In billing, a large unmeasured drop could mean rejected claims and compliance issues.

Check your understanding · explain the design in your own words

What proof is needed before merging focused pipeline steps into one prompt?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Faster pipelines: measure quality before merging steps Study: Decision rule: keep the focused stages until the candidate is measured

Study: Faster pipelines: measure quality before merging steps Study: Decision rule: keep the focused stages until the candidate is measured Accuracy-latency tradeoff Eval Golden set

Return to Q249's situation and clause-by-clause explanation

Drill · Claude Code (Q250–Q274)

Q250. A legal-research group runs Claude Code sessions that last most of a working day. Auto-compaction regularly kicks in, and afterwards the case citations Claude gathered early on are gone from the conversation. The group wants a script that pulls every citation out of the transcript at the exact point before anything is summarised, whether a person typed the command or the context limit triggered it. On which hook event should the script be registered? A) Stop, because it fires each time Claude finishes a response and can therefore scan the transcript frequently B) SessionEnd, because it fires once the session closes and can read the complete transcript in one pass C) UserPromptSubmit, filtering for prompts whose text starts with the compaction slash command D) PreCompact with no matcher, so it runs before both manual and automatic compaction and receives the transcript path

Understand Q250 · the situation and each part of the question

The real situation behind this question

A small legal-research group uses Claude Code as a research assistant over long days of case review. Early in each session Claude collects dozens of citations while reading briefs. By mid-afternoon the context fills and Claude Code compacts the conversation on its own. The summary keeps the thread of the argument but drops many exact citation strings, which the associates then have to find again by hand. They want a reliable, automatic capture of citations at the only moment it is still possible: just before summarisation, regardless of who or what triggered it.

Read the question in parts

Part 1 · at the exact point before anything is summarised

The hook must fire before compaction runs, not after the session or on unrelated events.

Part 2 · whether a person typed the command or the context limit triggered it

Both manual and automatic compaction must be covered, so a filter on typed prompts is not enough.

What you are being asked to decide

Which hook runs right before Claude Code summarises a long conversation? Match a lifecycle moment to the hook event that fires at that moment.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
PreCompactthe hook event that runs just before the conversation is summarised, manual or automatic
auto-compactionsummarisation Claude Code triggers itself when the context window fills

Study: Hooks: deterministic control inside Claude Code hook compaction context window

Understand Q250's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Compaction is not a tool call and not the end of a turn; it is its own lifecycle event. PreCompact fires before both manual and automatic compaction with the transcript path in hand, so it is the one place to capture detail that summarisation will drop.

Connect the design to the situation

An all-day research session loses early citations when auto-compaction runs.

Best answer: D. PreCompact is the lifecycle event that fires immediately before compaction, for both the manual command and the automatic trigger, and its input carries the transcript path. SessionEnd sounds complete but runs hours after the mid-session compaction has already discarded the early detail from the conversation.

What each choice would do here

A · Why this does not meet the full requirement

Stop, because it fires each time Claude finishes a response and can therefore scan the transcript frequently

Stop fires on every response, not at compaction, so it is noisy and not tied to the moment that matters.

A Stop hook would fire after every single response Claude gives, dozens of times a day. It could scan the transcript each time, but it has no idea when compaction is about to happen and produces constant redundant work.

B · Why this does not meet the full requirement

SessionEnd, because it fires once the session closes and can read the complete transcript in one pass

SessionEnd runs after the whole session, long after mid-session compaction already happened.

SessionEnd would fire once at the end of the day. By then the in-session compaction already happened, so the transcript file may still exist, but the requirement was capture at the moment before summarisation, and the running session never benefits.

C · Why this does not meet the full requirement

UserPromptSubmit, filtering for prompts whose text starts with the compaction slash command

UserPromptSubmit only sees typed prompts, so the automatic trigger is missed entirely.

A UserPromptSubmit filter catches only prompts a person types. Automatic compaction triggered by the context limit never passes through a typed prompt, so the most common case in this workflow would be silently missed.

D · Why this meets the requirement

PreCompact with no matcher, so it runs before both manual and automatic compaction and receives the transcript path

PreCompact runs right before compaction for both triggers and gets the transcript path, which is exactly the required moment.

Registering on PreCompact without a matcher covers both the manual and auto triggers. The script receives the transcript path in its input and can copy or parse the full pre-summary conversation before any detail is lost.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Register a PreCompact hook in .claude/settings.json

JSON
{
  "hooks": {
    "PreCompact": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "python3 .claude/hooks/extract_citations.py"
          }
        ]
      }
    ]
  }
}

No matcher means the hook runs for both the manual and auto triggers. The script reads the hook input on stdin, which includes transcript_path and trigger, and writes the citations to a file before summarisation.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Which hook runs right before Claude Code summarises a long conversation?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Hooks: deterministic control inside Claude Code

Study: Hooks: deterministic control inside Claude Code hook compaction context window

Return to Q250's situation and clause-by-clause explanation

Q251. A data-science platform requires every Python file to use a 100-character line limit, and the build fails on anything else. The rule sits in the committed project CLAUDE.md. A contractor's personal instruction file says "wrap at 79 characters", and several of the contractor's sessions have produced files that failed the build. Which statement about fixing this is exactly right? A) Adding a lineLength key to the committed .claude/settings.json makes Claude Code enforce the width on every generated file B) Any CLAUDE.md level is guidance the model weighs; a PostToolUse hook on Write and Edit that runs the formatter makes the width deterministic C) Project CLAUDE.md is loaded after the personal file, so it deterministically overrides it and the failures must come from something else D) Moving the rule into an organisation-wide managed CLAUDE.md guarantees compliance because users cannot override managed instructions

Understand Q251 · the situation and each part of the question

The real situation behind this question

A data-science platform team runs a strict build that rejects Python files over 100 characters wide. The rule has lived in the project CLAUDE.md for months and most sessions respect it. A contractor joined with a personal instruction file from a previous job that says to wrap at 79. In some sessions the model follows the personal line, the resulting files fail the build, and review time is wasted. The team lead wants a fix that does not depend on which instruction the model happens to weigh more in a particular session.

Read the question in parts

Part 1 · the build fails on anything else

This is a hard requirement with a real cost, so a best-effort instruction is not an acceptable answer.

Part 2 · A contractor's personal instruction file says

Two instruction files conflict, which shows that instruction placement alone cannot decide the outcome reliably.

Part 3 · Which statement about fixing this is exactly right?

Pick the statement whose mechanism is accurate, not merely a change that might reduce failures.

What you are being asked to decide

How do you guarantee a formatting rule when instruction files disagree? Separate guidance (instruction files at any level) from enforcement (hooks that run code).

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
PostToolUsea hook that runs your command after a tool such as Write or Edit succeeds
deterministicthe formatter produces the same result regardless of what the model chose
managed CLAUDE.mdan organisation-level instruction file; still loaded as context, not executed

Study: Team instructions versus enforced rules Study: What hooks guarantee and what they do not CLAUDE.md hook deterministic

Understand Q251's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Instruction files at any level, even managed ones, are context the model weighs, so a conflicting personal file can still win. When a rule must hold, move the guarantee into code that the harness runs: a PostToolUse formatter hook for Claude's writes, backed by CI for everything else.

Connect the design to the situation

A personal instruction file fights the project line-length rule and breaks the build.

Best answer: B. CLAUDE.md files at every level, including managed ones, are loaded into context as instructions, so conflicting guidance can still win in a given session. A PostToolUse hook runs the formatter as code after each Write or Edit, so the width no longer depends on which instruction the model followed. The managed-file answer is the tempting one, but it only raises the priority of advice; it does not execute anything.

What each choice would do here

A · Why this does not meet the full requirement

Adding a lineLength key to the committed .claude/settings.json makes Claude Code enforce the width on every generated file

There is no settings key that formats generated code; settings configure the runtime, not code style.

Claude Code settings hold permissions, environment variables, hooks and model choices. There is no line-length or code-style key, so this would be silently ignored while the team believes the problem is solved.

B · Why this meets the requirement

Any CLAUDE.md level is guidance the model weighs; a PostToolUse hook on Write and Edit that runs the formatter makes the width deterministic

Instruction files advise; a formatter hook after each write deterministically fixes the width for Claude's edits.

This names both halves correctly. Instruction files are guidance at every level. A PostToolUse hook matching Write and Edit runs the formatter on the changed file each time, so Claude's output is reformatted to 100 characters regardless of which instruction won; CI still covers edits made outside Claude.

C · Why this does not meet the full requirement

Project CLAUDE.md is loaded after the personal file, so it deterministically overrides it and the failures must come from something else

Load order does not turn guidance into a guarantee; conflicting instructions can still win.

This claims load order produces a deterministic override. Project and user files are both placed in context; more specific files tend to carry more weight, but the model still reads both, and the observed build failures prove the conflict is not resolved.

D · Why this does not meet the full requirement

Moving the rule into an organisation-wide managed CLAUDE.md guarantees compliance because users cannot override managed instructions

A managed instruction file is still context for the model, so priority rises but nothing is enforced.

A managed CLAUDE.md is deployed by administrators and cannot be excluded by users, but it is still text in context. It may lower the failure rate, yet the requirement is that no failing file reaches the build, which advice cannot promise.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Formatter hook registered in committed project settings

JSON
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write|Edit",
        "hooks": [
          {
            "type": "command",
            "command": "jq -r '.tool_input.file_path' | xargs ruff format --line-length 100"
          }
        ]
      }
    ]
  }
}

The hook reads the edited file path from the hook input and runs the formatter on it. The instruction in CLAUDE.md can stay as an explanation; the hook is what makes the rule hold.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you guarantee a formatting rule when instruction files disagree?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Team instructions versus enforced rules Study: What hooks guarantee and what they do not

Study: Team instructions versus enforced rules Study: What hooks guarantee and what they do not CLAUDE.md hook deterministic

Return to Q251's situation and clause-by-clause explanation

Q252. A mobile team's CLAUDE.md says "new screens must support accessibility." Claude Code keeps producing screens where a single icon has a content description, while buttons have no labels and touch targets are too small. Tests and CI are already configured. What is the root cause of the thin results? A) The instruction names a goal but no observable criteria, so Claude meets it minimally; it needs checkable requirements that tests can verify B) The instruction must live in a .claude/rules file to become binding, since CLAUDE.md content is treated as optional background C) CLAUDE.md is only consulted for files at the repository root, so screen files in nested folders never see the instruction D) Claude Code drops CLAUDE.md after each compaction, so the instruction fades as the session grows longer

Understand Q252 · the situation and each part of the question

The real situation behind this question

A mobile team building a banking app added one line to its project instructions: new screens must support accessibility. Claude Code picks the line up in every session, and each new screen does contain some accessibility work, usually one described icon. Buttons still lack labels and several tap areas are below platform minimums. The team already has UI tests and a CI pipeline. The lead is trying to decide whether the instruction is being ignored or whether it simply never said what done looks like.

Read the question in parts

Part 1 · a single icon has a content description

Claude did act on the instruction, which points away from a loading failure and toward vague wording.

Part 2 · Tests and CI are already configured

Verification infrastructure exists, so the missing piece is criteria it can check, not new tooling.

What you are being asked to decide

Why does Claude do the bare minimum on a broad instruction? Diagnose an under-specified instruction versus a loading or enforcement problem.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
acceptance criteriaspecific, checkable outcomes a screen must meet before it counts as done
CLAUDE.mdproject instructions loaded into every session as guidance

Study: Three mechanisms, three different jobs Study: CLAUDE.md: the memory hierarchy CLAUDE.md eval

Understand Q252's answer · full reasoning and worked examples
Claude synthesized · the point, straight

If Claude acts on an instruction but only barely, the instruction lacks observable criteria. Specific, checkable requirements improve what Claude produces, and the same criteria turned into tests or CI checks are what actually confirm the requirement is met.

Connect the design to the situation

"Support accessibility" yields one labelled icon and nothing else.

Best answer: A. Claude is following the instruction, just at the lowest bar the words allow, because "support accessibility" states no testable outcome. Rewriting it as concrete checks (every interactive element labelled, minimum touch target size, an automated accessibility test per screen) gives Claude a target and gives CI something to verify. Moving the same vague sentence into a rules file changes where it loads, not how specific it is.

What each choice would do here

A · Why this meets the requirement

The instruction names a goal but no observable criteria, so Claude meets it minimally; it needs checkable requirements that tests can verify

A goal without criteria invites the minimum; specific checks plus tests fix both the output and its verification.

Rewriting the instruction as a checklist, such as labels on every interactive element, a minimum touch target, and an accessibility test per screen, gives Claude an unambiguous target. The same criteria become CI assertions, so the requirement is both clearer and verified.

B · Why this does not meet the full requirement

The instruction must live in a .claude/rules file to become binding, since CLAUDE.md content is treated as optional background

Rules files are also guidance; relocating a vague sentence does not make it specific.

A rules file without paths loads exactly like CLAUDE.md, and one with paths loads when matching files are touched. Either way it is still guidance; the vague sentence would produce the same minimal screens.

C · Why this does not meet the full requirement

CLAUDE.md is only consulted for files at the repository root, so screen files in nested folders never see the instruction

Project CLAUDE.md is loaded for the whole session, not only for root-level files.

Project CLAUDE.md at the repository root is loaded at session start and applies while Claude edits files anywhere in the tree. The screens do show partial accessibility work, which proves the instruction was read.

D · Why this does not meet the full requirement

Claude Code drops CLAUDE.md after each compaction, so the instruction fades as the session grows longer

Project CLAUDE.md is reloaded after compaction, and the problem appears in fresh sessions too.

After compaction Claude Code re-reads the project CLAUDE.md from disk, so the instruction does not fade. The thin screens also appear early in fresh sessions, which rules out a length-related cause.

Check your understanding · explain the design in your own words

Why does Claude do the bare minimum on a broad instruction?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Three mechanisms, three different jobs Study: CLAUDE.md: the memory hierarchy

Study: Three mechanisms, three different jobs Study: CLAUDE.md: the memory hierarchy CLAUDE.md eval

Return to Q252's situation and clause-by-clause explanation

Q253. A game studio keeps separate standards for shader code, level data files and localisation strings. A lead put each standard in its own file under .claude/rules/ so that only the relevant one would be in context while Claude works in that area. None of the files has frontmatter. What actually happens? A) All three files load at the start of every session, so every standard is in context regardless of what is edited B) Claude Code infers the scope from each file name and loads it only for folders with a matching name C) Each file stays dormant until a CLAUDE.md imports it, so none of the three standards reaches Claude D) Claude Code refuses to start and reports that every rules file requires a paths header

Understand Q253 · the situation and each part of the question

The real situation behind this question

A game studio's repository holds engine shaders, JSON level definitions, and localisation tables, and each area has its own style guide. The lead wanted to stop loading all three guides into every Claude Code session, both to save context and to stop shader advice leaking into translation work. She moved each guide into its own file under .claude/rules/ and assumed that was enough. A week later engineers notice the localisation guidance showing up while Claude edits shader code.

Read the question in parts

Part 1 · only the relevant one would be in context

The goal is conditional loading, which requires a mechanism that decides by file path.

Part 2 · None of the files has frontmatter

Without a paths list there is nothing telling Claude Code when to load each file, so default loading applies.

What you are being asked to decide

What does a rules file do if it has no paths header? Know the exact loading behaviour of path-scoped rules versus unscoped ones.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
paths frontmattera YAML list of globs at the top of a rules file that limits when it loads
YAMLthe header format used for the paths list

Study: .claude/rules: path-scoped conventions frontmatter yaml context window

Understand Q253's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Putting guidance in .claude/rules/ only scopes it when the file declares paths globs. Without that frontmatter a rules file loads every session like CLAUDE.md, so selective loading needs an explicit glob per standard.

Connect the design to the situation

Three studio standards all load in every session because nobody added paths.

Best answer: A. A rules file without paths frontmatter is loaded unconditionally at session start, with the same standing as project CLAUDE.md. To load a standard only for its area, each file needs a paths list of globs such as shaders/** so it activates when Claude works with matching files. The file-name inference answer describes the desired behaviour, but Claude Code does not infer scope from names.

What each choice would do here

A · Why this meets the requirement

All three files load at the start of every session, so every standard is in context regardless of what is edited

Without paths, a rules file loads every session, so the scoping goal is not met.

With no paths condition, each rules file is treated like project memory and loaded at launch. Every session therefore carries all three standards. Adding paths such as shaders/**, levels/**/*.json and localisation/** restores the intended selective loading.

B · Why this does not meet the full requirement

Claude Code infers the scope from each file name and loads it only for folders with a matching name

Scope comes only from the paths globs, never from the file name.

Naming a file shaders.md does not scope it to a shaders folder. Claude Code never guesses scope from names; only explicit globs in the paths field restrict when a rule applies.

C · Why this does not meet the full requirement

Each file stays dormant until a CLAUDE.md imports it, so none of the three standards reaches Claude

Rules files are discovered automatically; no import is needed for them to load.

Imports are a CLAUDE.md feature for pulling in other files, but rules files do not depend on them. Claude Code discovers every file in .claude/rules/ on its own, so the standards do reach Claude, just not selectively.

D · Why this does not meet the full requirement

Claude Code refuses to start and reports that every rules file requires a paths header

Frontmatter is optional; missing paths changes scope, it does not cause an error.

Claude Code starts normally. Frontmatter is optional in rules files; its absence simply means the file has no path condition. No error or warning is raised, which is why the lead did not notice.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

A scoped rules file for shader code

TEXT
---
paths:
  - "shaders/**"
  - "**/*.hlsl"
---
# Shader standards
- Use half precision for colour values unless a comment justifies float.
- Every shader declares its target profile at the top.

The paths list is what makes this file conditional; without it the same file would be loaded in every session.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What does a rules file do if it has no paths header?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: .claude/rules: path-scoped conventions

Study: .claude/rules: path-scoped conventions frontmatter yaml context window

Return to Q253's situation and clause-by-clause explanation

Q254. A Go team registers gofmt -w on the edited file as a PreToolUse hook matching Edit and Write, expecting every file Claude touches to end up formatted. After a week, files Claude modified are still unformatted in review. What is the cause? A) Formatting is only applied when the requirement is written in CLAUDE.md so that Claude runs the tool itself B) Hooks cannot launch external binaries, so gofmt is skipped and only built-in checks run C) The matcher must be Bash because formatters are shell commands, so an Edit matcher never fires D) PreToolUse runs before the edit is applied, so gofmt formats the old file and Claude's change lands afterwards

Understand Q254 · the situation and each part of the question

The real situation behind this question

A Go backend team wanted formatting to stop being a review topic. An engineer added a hook that runs gofmt on whatever file Claude edits, and chose PreToolUse because it sounded like the place to prepare a file. Nobody tested it on a real edit. A week later reviewers still see mixed indentation and import order in files Claude changed, and the team wonders whether hooks are reliable at all or whether the configuration is wrong.

Read the question in parts

Part 1 · as a PreToolUse hook matching Edit and Write

The hook timing is before the tool runs, which is the detail that decides what file content it sees.

Part 2 · files Claude modified are still unformatted

The hook runs, but its effect is overwritten, which points to timing rather than a missing hook.

What you are being asked to decide

Why does a formatter hook set to run before edits leave files unformatted? Pick the hook timing that matches the job: block before, fix after.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
PreToolUseruns before the tool call; suited to allowing or blocking it
PostToolUseruns after the tool call succeeds; suited to formatting or checking the result

Study: Hooks: deterministic control inside Claude Code Study: What hooks guarantee and what they do not hook deterministic

Understand Q254's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Hook timing decides what the hook sees. PreToolUse can inspect or block a call before it happens, but a formatter there works on the stale file; fixing what Claude wrote requires PostToolUse.

Connect the design to the situation

gofmt runs as PreToolUse and formats the stale file, not Claude's change.

Best answer: D. PreToolUse fires before the tool executes, so the formatter rewrites the previous version on disk and the new content from Edit or Write then replaces it unformatted. PostToolUse fires after the write succeeds, which is when a formatter can act on what Claude actually produced. Putting the requirement in CLAUDE.md would only ask Claude to remember, which is the exact gap the hook was meant to close.

What each choice would do here

A · Why this does not meet the full requirement

Formatting is only applied when the requirement is written in CLAUDE.md so that Claude runs the tool itself

An instruction is guidance; the hook exists to avoid relying on Claude remembering.

Asking Claude to run gofmt through CLAUDE.md makes formatting depend on the model remembering every time. The whole point of a hook is that the harness runs the command whether or not the model thinks of it.

B · Why this does not meet the full requirement

Hooks cannot launch external binaries, so gofmt is skipped and only built-in checks run

Command hooks run any shell command, including formatters.

Command hooks execute arbitrary shell commands with the user's permissions, so gofmt runs fine. That is precisely why the team saw no error: the formatter ran successfully, just on the wrong version of the file.

C · Why this does not meet the full requirement

The matcher must be Bash because formatters are shell commands, so an Edit matcher never fires

Matchers name the tool Claude uses; Edit and Write are valid matchers for file changes.

Matchers match the tool Claude is calling, not the kind of command the hook runs. Edit and Write are exactly the tools that change files; a Bash matcher would only fire when Claude runs shell commands.

D · Why this meets the requirement

PreToolUse runs before the edit is applied, so gofmt formats the old file and Claude's change lands afterwards

Before-the-edit timing formats stale content; the new write then overwrites it.

At PreToolUse time the file on disk is the old version, or does not exist yet for a new Write. gofmt formats that, then Edit or Write applies Claude's new content. Moving the same command to PostToolUse makes it format the result.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Correct timing: format after the write

JSON
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [
          {
            "type": "command",
            "command": "jq -r '.tool_input.file_path' | xargs gofmt -w"
          }
        ]
      }
    ]
  }
}

Same command, different event. PostToolUse receives the path of the file just written, so gofmt acts on Claude's actual output.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why does a formatter hook set to run before edits leave files unformatted?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Hooks: deterministic control inside Claude Code Study: What hooks guarantee and what they do not

Study: Hooks: deterministic control inside Claude Code Study: What hooks guarantee and what they do not hook deterministic

Return to Q254's situation and clause-by-clause explanation

Q255. At a fintech company, two developers each start a Claude Code session in the same checkout: one upgrades the logging library, the other adds a new report endpoint on a different branch. Within an hour one session's test run picks up the other's half-written files, and a branch switch in one session changes the files the other is editing. What is the underlying cause and fix? A) Claude Code does not support concurrent sessions; run the two tasks one after the other in a single session B) The two sessions share a single context window; clear the conversation in each session before switching tasks C) Both sessions lack a lock file in settings; enable file locking so edits from one session are hidden from the other D) Both sessions share one working directory; give each its own git worktree on its own branch and run one session per worktree

Understand Q255 · the situation and each part of the question

The real situation behind this question

Two developers at a fintech company wanted to move faster by running Claude Code sessions side by side on the same repository. One asked Claude to upgrade the logging library across services, the other to add a reporting endpoint. They both opened sessions in the same folder. Soon one session's tests failed on code the other had only half written, and when one checked out a different branch, the other session's open edits vanished from disk. They need both tasks running at once without interference.

Read the question in parts

Part 1 · in the same checkout

Both sessions read and write the same files on disk, which is the shared resource causing collisions.

Part 2 · a branch switch in one session changes the files the other is editing

Branches alone cannot isolate sessions when they share a single working directory.

What you are being asked to decide

Why do two Claude sessions in one folder trip over each other? Identify filesystem isolation as the fix for parallel sessions.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
git worktreea second working directory for the same repository, checked out on its own branch
sessionone running Claude Code conversation with its own context

Study: Separate checkouts and handoffs Study: Plan mode vs direct execution and the Explore subagent worktree isolation Claude Code

Understand Q255's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Parallel sessions have separate conversations but not separate files. When they share a checkout, edits, tests and branch switches collide; a worktree per session gives each its own directory and branch, leaving integration to a normal merge.

Connect the design to the situation

Two sessions in one checkout see each other's half-written files.

Best answer: D. The sessions have independent conversations but share one set of files, so edits, test runs and branch switches collide. A git worktree gives each session its own directory and branch from the same repository, which isolates the files; merging the branches later is still normal integration work. Running the tasks one after another would avoid the collision but gives up the parallelism the team wanted.

What each choice would do here

A · Why this does not meet the full requirement

Claude Code does not support concurrent sessions; run the two tasks one after the other in a single session

Concurrent sessions are supported; sequencing only avoids the need for isolation.

Running the tasks in sequence would prevent the interference, but Claude Code supports many concurrent sessions and the team explicitly wants parallel work. This trades away the goal instead of isolating the sessions.

B · Why this does not meet the full requirement

The two sessions share a single context window; clear the conversation in each session before switching tasks

Sessions already have separate contexts; the shared resource is the directory.

Each Claude Code session already has its own conversation and context window, so clearing it changes nothing about the files. The collision comes from the shared directory, which neither session's context controls.

C · Why this does not meet the full requirement

Both sessions lack a lock file in settings; enable file locking so edits from one session are hidden from the other

There is no settings option that hides one session's file edits from another.

Claude Code settings have no file-locking or edit-hiding feature between sessions. Files on disk are shared by definition; only separate directories keep one session's edits away from the other.

D · Why this meets the requirement

Both sessions share one working directory; give each its own git worktree on its own branch and run one session per worktree

Separate worktrees give each session separate files and a separate branch.

Running git worktree add for each task creates a separate directory with its own branch, backed by the same repository. Each session edits and tests only its own files; integration happens later through a normal merge or pull request.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

One worktree per task

TEXT
git worktree add ../app-logging -b chore/logging-upgrade
git worktree add ../app-report -b feat/report-endpoint
cd ../app-logging && claude   # session one
cd ../app-report && claude    # session two

Each session starts in its own directory on its own branch. Merge conflicts can still appear at integration time; worktrees prevent live interference, not every conflict.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why do two Claude sessions in one folder trip over each other?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Separate checkouts and handoffs Study: Plan mode vs direct execution and the Explore subagent

Study: Separate checkouts and handoffs Study: Plan mode vs direct execution and the Explore subagent worktree isolation Claude Code

Return to Q255's situation and clause-by-clause explanation

Q256. An API rename means the client docs for four language SDKs (Python, Go, Java, Ruby) must all be rewritten, and each rewrite requires reading that SDK's source in depth. Holding all four codebases in one conversation would exceed the context window, and the work should run at the same time. Which plan meets both constraints? A) Run one session over all four SDKs in turn and compact the conversation between each language B) Start four sessions in the same checkout, each switching to its own branch before it begins editing C) Create a git worktree per SDK and run one Claude Code session in each, then merge the four branches D) Create one skill with forked context that walks through all four SDKs inside a single forked run

Understand Q256 · the situation and each part of the question

The real situation behind this question

A developer-platform team renamed several API endpoints, which means the reference docs for four SDKs are now wrong. Each SDK has its own idioms, and documenting it correctly means reading its client code, examples and tests. A trial run in a single Claude Code session filled the context before the second SDK was finished. The release is in two days, so the team wants all four rewrites running at once, with each one reading its own SDK thoroughly and none stepping on another's files.

Read the question in parts

Part 1 · would exceed the context window

One conversation cannot hold everything, so the work must be divided across separate contexts.

Part 2 · the work should run at the same time

Parallel sessions will edit the repository simultaneously, so each needs its own working files.

What you are being asked to decide

How do you split a job that is too big for one context and must run in parallel? Recognise that context isolation and file isolation are separate needs.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
context isolationeach task gets its own conversation so one task's reading does not crowd another
worktreea separate working directory and branch for one session

Study: Separate checkouts and handoffs Study: Skill frontmatter options worktree isolation context isolation context window

Understand Q256's answer · full reasoning and worked examples
Claude synthesized · the point, straight

This job needs two different kinds of isolation. Separate sessions stop one conversation from having to hold every codebase, and separate worktrees stop parallel sessions from editing the same files; either one alone leaves one of the constraints unmet.

Connect the design to the situation

Four SDK doc rewrites, each needing deep reading of its own codebase.

Best answer: C. Four sessions give four separate context windows, so no single conversation has to hold every codebase, and four worktrees give each session its own files and branch so they can edit at the same time. A single forked skill does isolate its output from the main conversation, but it is still one context that must read all four SDKs.

What each choice would do here

A · Why this does not meet the full requirement

Run one session over all four SDKs in turn and compact the conversation between each language

Serial and lossy: compaction drops detail and nothing runs in parallel.

One session doing four SDKs in turn is serial by design, and each compaction replaces detail with a summary, so later rewrites lose facts from earlier reading. It addresses neither the parallel requirement nor the quality of the reading.

B · Why this does not meet the full requirement

Start four sessions in the same checkout, each switching to its own branch before it begins editing

Four contexts, but one shared directory, so branch switches collide.

Four sessions do give four separate contexts, solving the size problem. Sharing one checkout means the branch checkouts overwrite each other's files on disk, so the parallel edits collide and tests see mixed states.

C · Why this meets the requirement

Create a git worktree per SDK and run one Claude Code session in each, then merge the four branches

Separate sessions split the context load; separate worktrees keep their files apart.

Each worktree is its own directory on its own branch, and each session has its own context focused on one SDK. The four rewrites run concurrently without file collisions, and the branches are merged afterwards like any other change.

D · Why this does not meet the full requirement

Create one skill with forked context that walks through all four SDKs inside a single forked run

A forked skill is still one context that has to read all four codebases.

A skill with forked context runs in a separate subagent context, which keeps its verbose output out of the main conversation. But it is one fork reading all four codebases, so it hits the same context ceiling and runs nothing in parallel.

Check your understanding · explain the design in your own words

How do you split a job that is too big for one context and must run in parallel?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Separate checkouts and handoffs Study: Skill frontmatter options

Study: Separate checkouts and handoffs Study: Skill frontmatter options worktree isolation context isolation context window

Return to Q256's situation and clause-by-clause explanation

Q257. A health-records team requires every new database migration to be named V__.sql. Claude Code usually complies, but about one migration in ten gets a name like V3__AddTable.sql, which breaks the deploy tool. A teammate proposes adding three good and three bad examples to the project CLAUDE.md. What change actually guarantees non-conforming names never land? A) Create a .claude/rules file scoped to the migrations folder that restates the naming pattern with examples B) Add a PreToolUse hook on Write that checks the file path against the pattern and exits with code 2 and a reason C) Add the six examples to CLAUDE.md with the heading IMPORTANT so the model weighs them above other guidance D) Ask a review subagent at the end of each session to list any migrations whose names do not match the pattern

Understand Q257 · the situation and each part of the question

The real situation behind this question

A health-records team deploys schema changes with a tool that orders migrations by timestamp in the file name. The naming rule is written in the project instructions, and Claude Code follows it most of the time. Roughly one migration in ten comes out with a short sequential name, the deploy tool rejects it, and a release stalls while someone renames the file. A teammate wants to add more examples to the instructions. The lead asks whether any wording change can make the rule hold every time.

Read the question in parts

Part 1 · about one migration in ten

The rule is already understood; the failures are occasional, which more guidance cannot eliminate.

Part 2 · guarantees non-conforming names never land

The requirement is prevention every time, which needs a deterministic check before the file is written.

What you are being asked to decide

How do you make sure a file-naming rule is never broken? Choose enforcement code over stronger instructions for a checkable rule.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
exit code 2the hook result that blocks the tool call and sends the message back to Claude
mechanically checkablea rule a short script can verify with no judgment involved
guardraila control that stops a bad action rather than hoping it is avoided

Study: Team instructions versus enforced rules Study: Hooks: deterministic control inside Claude Code hook deterministic guardrail

Understand Q257's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When a rule can be checked by a script, occasional misses are an enforcement problem, not a wording problem. A PreToolUse hook that tests the file name and blocks with a reason makes the rule hold every time, while instructions stay as explanation.

Connect the design to the situation

One migration in ten gets a wrong name despite clear instructions.

Best answer: B. A PreToolUse hook runs as code before every Write, so a name that fails the regular expression is blocked and the reason is returned to Claude, which can retry with a valid name. Examples in CLAUDE.md or a scoped rules file improve the odds but remain instructions the model can miss one time in ten. A review subagent detects problems only after the files exist, and its check is itself model judgment.

What each choice would do here

A · Why this does not meet the full requirement

Create a .claude/rules file scoped to the migrations folder that restates the naming pattern with examples

A scoped rules file is guidance that loads at the right time, not a check.

Scoping the reminder to migrations keeps it relevant and out of unrelated sessions. It is loaded as context when Claude works with those files, so it is the same kind of guidance as CLAUDE.md, just better targeted.

B · Why this meets the requirement

Add a PreToolUse hook on Write that checks the file path against the pattern and exits with code 2 and a reason

Code runs before every write and blocks bad names, returning the reason to Claude.

The hook receives the tool input, including the file path, before the write happens. A script tests the name against the pattern and exits with code 2 plus a message when it fails, so the write is blocked and Claude gets a precise reason to correct the name.

C · Why this does not meet the full requirement

Add the six examples to CLAUDE.md with the heading IMPORTANT so the model weighs them above other guidance

Louder instructions are still instructions; the one-in-ten miss can recur.

Emphasis and examples genuinely raise compliance, and they are worth keeping as explanation. But the output is still a model choice, so the occasional bad name remains possible and the deploy tool will still break on it.

D · Why this does not meet the full requirement

Ask a review subagent at the end of each session to list any migrations whose names do not match the pattern

Detective and after the fact, and the check relies on another model's judgment.

An end-of-session review would report bad names after they exist, and a model-based reviewer can itself miss a case. It is a detective control at the wrong time, useful for audits but not for a guarantee.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Hook script that blocks a bad migration name

PYTHON
import json, re, sys

event = json.load(sys.stdin)
path = event.get("tool_input", {}).get("file_path", "")
name = path.rsplit("/", 1)[-1]
if "/migrations/" in path and not re.fullmatch(r"V\d{12}__[a-z0-9_]+\.sql", name):
    print(f"Migration name {name} must match V<yyyymmddHHMM>__<snake_case>.sql", file=sys.stderr)
    sys.exit(2)
sys.exit(0)

Registered as a PreToolUse hook on Write. Exit code 2 blocks the call and the stderr text is shown to Claude, which retries with a valid name.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you make sure a file-naming rule is never broken?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Team instructions versus enforced rules Study: Hooks: deterministic control inside Claude Code

Study: Team instructions versus enforced rules Study: Hooks: deterministic control inside Claude Code hook deterministic guardrail

Return to Q257's situation and clause-by-clause explanation

Q258. Before changing how permissions are checked in a large Rails monolith, a developer asks Claude Code how authorization works. The logic is spread across concerns, mixins, callbacks and a policy layer in roughly thirty files. Claude opens two files and gives a confident but partial answer. Which approach gives a thorough answer without filling the main conversation with file dumps? A) Paste the full list of thirty file paths into the prompt and ask Claude to read all of them in this session B) Run the compaction command first so that the main conversation has more room for the files it reads C) Enable extended thinking so Claude reasons more carefully about the two files it has already read D) Delegate the tracing to the Explore subagent, asking it to follow every call path and return a summary

Understand Q258 · the situation and each part of the question

The real situation behind this question

A developer at a subscription-software company needs to change how access checks work, but the authorization logic has grown over ten years. It lives in model concerns, controller callbacks, mixins and a separate policy layer. Asking Claude Code for an explanation produced a tidy answer based on two files that missed the callback chain entirely. The developer wants the full picture before planning a change, and also wants to keep the working conversation focused on that change rather than on thirty pasted files.

Read the question in parts

Part 1 · spread across concerns, mixins, callbacks and a policy layer

Understanding requires reading widely across many files, which is verbose discovery work.

Part 2 · without filling the main conversation with file dumps

The reading should happen somewhere other than the main context, with only findings returned.

What you are being asked to decide

How do you get a deep code walkthrough without flooding your session? Pick the delegation mechanism for wide, read-heavy discovery.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Explore subagenta built-in read-only helper that searches the codebase in its own context
context isolationverbose reading stays in the helper's context; only the summary returns

Study: Plan mode vs direct execution and the Explore subagent subagent context isolation Claude Code

Understand Q258's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Shallow answers on sprawling code come from shallow reading, and forcing deep reading into the main session trades one problem for another. The Explore subagent reads widely in its own context and returns a summary, giving depth without polluting the working conversation.

Connect the design to the situation

Authorization logic is scattered across thirty Rails files.

Best answer: D. The Explore subagent is a read-only helper that searches and reads across the codebase in its own context window and hands back a condensed result, so the investigation can be broad while the main conversation stays clean. Pasting all thirty paths would force a thorough read, but every file would land in the main context the developer needs for the change itself.

What each choice would do here

A · Why this does not meet the full requirement

Paste the full list of thirty file paths into the prompt and ask Claude to read all of them in this session

Thorough, but all thirty files crowd the main conversation.

Listing every file would make Claude read them all, so the answer would likely be complete. The cost is that all that raw code now sits in the main context, crowding out the planning and editing that follow.

B · Why this does not meet the full requirement

Run the compaction command first so that the main conversation has more room for the files it reads

Freeing space does not make the investigation broader or smarter.

Compacting first frees context space, but the problem was shallow reading, not lack of room. Claude would still stop after a couple of files unless the investigation itself is set up to be thorough.

C · Why this does not meet the full requirement

Enable extended thinking so Claude reasons more carefully about the two files it has already read

Deeper reasoning over two files cannot reveal code that was never read.

Extended thinking gives Claude more room to reason, but the answer was partial because the relevant files were never opened. More reasoning over the same two files will produce a more elaborate version of the same incomplete picture.

D · Why this meets the requirement

Delegate the tracing to the Explore subagent, asking it to follow every call path and return a summary

Wide reading happens in a separate context and returns a summary.

Claude Code can hand discovery to the Explore subagent, which is optimised for searching and reading without editing. It traces the callbacks and policies in its own context and returns a summary with file references, leaving the main conversation ready for planning.

Check your understanding · explain the design in your own words

How do you get a deep code walkthrough without flooding your session?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Plan mode vs direct execution and the Explore subagent

Study: Plan mode vs direct execution and the Explore subagent subagent context isolation Claude Code

Return to Q258's situation and clause-by-clause explanation

Q259. A nightly job runs claude -p with the prompt "Confirm every feature flag in flags.yaml is still referenced in the code" and a JSON output format. It reports "all flags in use" every night, yet engineers keep finding dead flags. What is the most likely cause? A) A single instance cannot check itself; a second headless instance must confirm every verdict the first one gives B) The prompt asks for a verdict with no required evidence, so Claude samples some flags and generalises; require each flag listed with the file and line where it is used C) The JSON output format truncates long answers, so the list of unused flags is cut off before it is printed D) Print mode runs without file tools, so Claude answers from the prompt text alone and never reads flags.yaml

Understand Q259 · the situation and each part of the question

The real situation behind this question

A consumer-app company uses feature flags heavily and wants to delete ones that no longer do anything. An engineer set up a nightly job that asks Claude Code, in headless mode, to confirm every flag is still referenced. Each morning the job reports that all flags are in use. Meanwhile engineers deleting code keep stumbling on flags with no references at all. The team is unsure whether the tool is broken, the output is being mangled, or the job simply asks the wrong question.

Read the question in parts

Part 1 · Confirm every feature flag in flags.yaml is still referenced

The prompt requests a confirmation, not a per-flag result with proof, so a quick yes satisfies it.

Part 2 · engineers keep finding dead flags

The verdict is wrong in a consistent direction, pointing to a shallow check rather than random failure.

What you are being asked to decide

Why does an automated check keep saying everything is fine when it is not? Diagnose missing validation criteria in a headless task.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
headlessrunning Claude Code non-interactively with -p, as in CI jobs
evidencethe concrete file and line proving each claim, so the result can be checked

Study: Claude Code in CI/CD headless eval deterministic

Understand Q259's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A check that returns only a verdict can be satisfied by a partial search. Requiring every item with concrete evidence makes the output complete and verifiable, and when the check is a plain string search, a deterministic script is the stronger design.

Connect the design to the situation

A nightly flag audit always reports zero unused flags.

Best answer: B. The prompt defines success as a yes-or-no statement, so a plausible verdict after a partial search satisfies it. Requiring a per-flag table with file and line evidence, and marking flags with no evidence as unused, turns the claim into something checkable; for a pure string search, a deterministic script is even better. A second instance reviewing the same vague verdict inherits the same missing criteria.

What each choice would do here

A · Why this does not meet the full requirement

A single instance cannot check itself; a second headless instance must confirm every verdict the first one gives

A second reviewer with the same vague criteria repeats the same gap.

Adding a reviewer instance doubles cost without adding criteria. If the reviewer is asked to confirm the same verdict, it can make the same sampling shortcut, so the dead flags remain invisible.

B · Why this meets the requirement

The prompt asks for a verdict with no required evidence, so Claude samples some flags and generalises; require each flag listed with the file and line where it is used

Without required evidence per item, a partial search can still produce a clean verdict.

Asked only to confirm, Claude can check a handful of flags, find them used, and conclude the rest are too. Requiring every flag in a table with a file and line, and treating a missing reference as unused, forces a complete pass and lets the job verify the output.

C · Why this does not meet the full requirement

The JSON output format truncates long answers, so the list of unused flags is cut off before it is printed

JSON output wraps the full result; it does not cut findings off.

The JSON output format returns the full result along with metadata such as cost and session id. It does not shorten the answer, and the answer here is short anyway because it contains only a verdict.

D · Why this does not meet the full requirement

Print mode runs without file tools, so Claude answers from the prompt text alone and never reads flags.yaml

Print mode still reads files; read-only tools do not need approval.

Headless print mode keeps the same tools as an interactive session. Read, Grep and Glob are read-only and run without approval, so Claude can open flags.yaml and search the code; the failure is in what it was asked to prove.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

A prompt that demands evidence

TEXT
claude -p "For EVERY key in flags.yaml, search the source tree.
Return JSON: [{\"flag\": ..., \"references\": [\"path:line\", ...]}].
A flag with an empty references list is UNUSED. Do not summarise; list every flag." --output-format json

The result now covers every flag and every claim is checkable; the job can also compare the flag count in the output with flags.yaml.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why does an automated check keep saying everything is fine when it is not?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Claude Code in CI/CD

Study: Claude Code in CI/CD headless eval deterministic

Return to Q259's situation and clause-by-clause explanation

Q260. Compliance at an insurance firm states that Claude Code must never open files under the secrets/ folder in any engineer's session. The rule is currently a sentence in the project CLAUDE.md, and one engineer's personal instruction file tells Claude to "read whatever you need to finish the task." Where should the rule live so that it holds in every session? A) In a CLAUDE.local.md at the repository root, because it is read last and overrides earlier instruction files B) In the committed .claude/settings.json as a deny rule on reading secrets/, backed by a hook for shell access C) In an organisation-wide managed CLAUDE.md, because personal instruction files cannot remove managed instructions D) In a .claude/rules file with paths set to secrets/, so the prohibition loads whenever those files come up

Understand Q260 · the situation and each part of the question

The real situation behind this question

An insurance firm keeps service credentials in a secrets folder that is excluded from most tooling. Compliance wants assurance that Claude Code never reads those files in any engineer's session. Today the rule is one sentence in the project instructions. An engineer who moved from a startup has a personal instruction file telling Claude to read whatever it needs. An audit found one session where Claude opened a credential file while debugging a connection error, and compliance now wants a rule that does not depend on which instruction wins.

Read the question in parts

Part 1 · must never open files under the secrets/ folder

This is an absolute prohibition, which requires a control that blocks the action rather than discourages it.

Part 2 · one engineer's personal instruction file tells Claude

Instructions already conflict, showing that any instruction-based rule can lose in some session.

Part 3 · holds in every session

The control must work regardless of personal configuration, which points to settings evaluated by the client.

What you are being asked to decide

Where do you put a rule that Claude must never break? Separate enforcement locations (settings, hooks) from guidance locations (every CLAUDE.md level).

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
deny rulea permission entry in settings that blocks a matching tool call before it runs
managed CLAUDE.mdan organisation-deployed instruction file; loaded as guidance, not enforced
CLAUDE.mdinstruction files at any level; the model reads and weighs them

Study: Where enforcement actually lives Study: Permissions and settings precedence settings-precedence CLAUDE.md managed policy

Understand Q260's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Instruction files are guidance at every level, including managed and path-scoped ones, so a competing instruction can still win. A rule that must hold goes where the client enforces it: permission deny rules in settings, plus a hook for paths those rules do not cover.

Connect the design to the situation

A personal instruction file competes with a no-secrets rule written in CLAUDE.md.

Best answer: B. A permission deny rule is evaluated by the Claude Code client before a tool runs, so the read is refused no matter what any instruction says, and deny rules from any settings level cannot be overridden by an allow elsewhere. Every CLAUDE.md variant, including managed and path-scoped ones, is text the model weighs. File deny rules cover Claude's file tools, so shell commands need their own deny rule or hook.

What each choice would do here

A · Why this does not meet the full requirement

In a CLAUDE.local.md at the repository root, because it is read last and overrides earlier instruction files

A local instruction file is still guidance, and it is personal and uncommitted.

CLAUDE.local.md is a personal, git-ignored instruction file, so it would not even reach other engineers. Even where it loads, it is text in context; the model can still weigh the engineer's broader instruction more heavily in a given moment.

B · Why this meets the requirement

In the committed .claude/settings.json as a deny rule on reading secrets/**, backed by a hook for shell access

The client enforces a deny rule before the tool runs, whatever any instruction says.

A deny rule such as Read(./secrets/**) in committed project settings is checked by the client before the tool call, and a deny at any settings level wins over allows. Because file rules govern Claude's file tools, a Bash deny rule or PreToolUse hook closes the shell path too.

C · Why this does not meet the full requirement

In an organisation-wide managed CLAUDE.md, because personal instruction files cannot remove managed instructions

Managed instructions cannot be excluded, but they are still advice the model weighs.

A managed instruction file is the most tempting wrong answer: administrators deploy it and users cannot exclude it. It is still loaded into context as guidance, so a debugging session can still decide that reading a credential file is necessary.

D · Why this does not meet the full requirement

In a .claude/rules file with paths set to secrets/**, so the prohibition loads whenever those files come up

Scoped guidance loads when secrets are touched, which is too late and still not enforced.

A path-scoped rule loads when Claude works with matching files, which is roughly the moment of the violation. It is also guidance, not a block, so the same session that decided to read the file can proceed.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Enforcement lives in settings, not prose

JSON
{
  "permissions": {
    "deny": [
      "Read(./secrets/**)",
      "Edit(./secrets/**)",
      "Bash(cat secrets/*)"
    ]
  }
}

Committed in .claude/settings.json, these rules are evaluated by the client for every engineer. For stronger coverage of shell access, add a PreToolUse hook on Bash that rejects any command mentioning secrets/, or deploy the rules through managed settings.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Where do you put a rule that Claude must never break?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Where enforcement actually lives Study: Permissions and settings precedence

Study: Where enforcement actually lives Study: Permissions and settings precedence settings-precedence CLAUDE.md managed policy

Return to Q260's situation and clause-by-clause explanation

Q261. A platform engineer wrote a release-notes skill that drafts notes from merged pull requests, and it works well on her laptop. Teammates who pulled the latest main branch report that it does not exist for them. She had saved it at ~/.claude/skills/release-notes/SKILL.md. What is the simplest change that gives it to everyone who clones this repository? A) Register the skill by name in the project settings file so that Claude Code knows to load it for all users B) Paste the skill body into the project CLAUDE.md so that it is loaded into every teammate's sessions C) Add a user-invocable line to the skill frontmatter so that it becomes visible to users other than its author D) Move it to .claude/skills/release-notes/SKILL.md inside the repository and commit that directory

Understand Q261 · the situation and each part of the question

The real situation behind this question

A platform engineer built a skill that reads merged pull requests and drafts release notes in the company's format. She tested it for a week and announced it in the team channel. Several teammates pulled main and tried it, but Claude Code did not recognise it. She had created it with the defaults on her own machine, in her home directory. The team wants every current and future engineer who clones the repository to have the same skill without any manual setup.

Read the question in parts

Part 1 · saved it at ~/.claude/skills/release-notes/SKILL.md

The skill lives in a personal location outside the repository, so version control never carries it.

Part 2 · everyone who clones this repository

The sharing mechanism must be the repository itself, which points to a committed project directory.

What you are being asked to decide

Where do you save a skill so the whole team gets it? Match skill scope to its storage location.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
project skilla skill stored in the repository's .claude/skills folder and shared through git
SKILL.mdthe file that defines a skill: frontmatter plus instructions

Study: Skills vs slash commands vs subagents skill agent-skills CLAUDE.md

Understand Q261's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Skill scope follows location: the home directory is personal, the repository's .claude/skills/ is shared. Committing the skill folder into the project is the simplest way to give it to every clone while keeping it on demand.

Connect the design to the situation

A skill in the home directory is invisible to teammates.

Best answer: D. Skills in ~/.claude/skills are personal and live only in that user's home directory, so a git pull can never deliver them. Project skills in .claude/skills/ inside the repository are discovered automatically for anyone who clones it once the directory is committed. Pasting the body into CLAUDE.md shares the text but turns an on-demand procedure into always-loaded context.

What each choice would do here

A · Why this does not meet the full requirement

Register the skill by name in the project settings file so that Claude Code knows to load it for all users

Skills are discovered from their folders; no settings registration exists or is needed.

Claude Code finds skills by scanning the personal and project skills directories. There is no settings key that lists skills by name, so this edit would do nothing and the skill would still be missing for teammates.

B · Why this does not meet the full requirement

Paste the skill body into the project CLAUDE.md so that it is loaded into every teammate's sessions

Shares the text but loads it every session instead of on demand.

Everyone would receive the text, but a skill's value is that its full instructions load only when invoked. Putting a long procedure in CLAUDE.md spends context in every session and removes the clean slash-command entry point.

C · Why this does not meet the full requirement

Add a user-invocable line to the skill frontmatter so that it becomes visible to users other than its author

Visibility to other users is about location, not a frontmatter flag.

Frontmatter can control whether a user can invoke a skill from the slash menu, and it is on by default. It has nothing to do with other people's machines, which cannot see a file in her home directory.

D · Why this meets the requirement

Move it to .claude/skills/release-notes/SKILL.md inside the repository and commit that directory

Committed project skills load for anyone who clones the repository.

Moving the folder into the repository's .claude/skills/ and committing it makes it a project skill. Every clone then has the file, Claude Code discovers it automatically, and changes are reviewed like any other code.

Check your understanding · explain the design in your own words

Where do you save a skill so the whole team gets it?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Skills vs slash commands vs subagents

Study: Skills vs slash commands vs subagents skill agent-skills CLAUDE.md

Return to Q261's situation and clause-by-clause explanation

Q262. A consultant opens Claude Code inside a client's repository, which is cloned under a shared workspace folder. Claude starts insisting on a changelog entry for every change, and nothing in the repository says so. The consultant suspects an instruction file somewhere above the repository. Which command answers this directly inside the running session? A) /doctor, which checks the installation and reports problems with the Claude Code setup B) /memory, which lists the instruction files loaded for the session and lets you open each one C) /clear, which empties the conversation so the session starts again without inherited instructions D) /init, which scans the repository and writes a CLAUDE.md describing the conventions it finds

Understand Q262 · the situation and each part of the question

The real situation behind this question

A consultant is brought in to fix a performance issue in a client's service. The client's repositories are all cloned under one shared workspace folder on a jump box. As soon as the consultant starts Claude Code, it insists on writing a changelog entry for every edit, which nobody asked for and which the repository's own instructions never mention. The consultant does not want to hunt through folders by hand and wants a direct answer from inside the session.

Read the question in parts

Part 1 · nothing in the repository says so

The instruction must be coming from a file outside the repository, such as a parent folder or user file.

Part 2 · inside the running session

The answer should come from a session command that inspects what is actually loaded right now.

What you are being asked to decide

How do you see which instruction files a session is using? Pick the diagnostic that reveals loaded memory files.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
memory filesthe CLAUDE.md and related instruction files loaded into a session
parent directorya folder above the repository whose CLAUDE.md is still loaded

Study: Which instructions are loaded Study: CLAUDE.md: the memory hierarchy CLAUDE.md Claude Code

Understand Q262's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When behaviour comes from an unknown instruction, look at what the session actually loaded. /memory lists every loaded memory file, including ones in parent folders, which turns guessing into a direct answer.

Connect the design to the situation

An unexplained rule comes from a CLAUDE.md in a parent folder.

Best answer: B. /memory shows the memory files the session has loaded, including CLAUDE.md files found in parent directories, and opens any of them for editing, so the consultant can see exactly which file introduces the changelog rule. /init creates a new CLAUDE.md from the codebase, which adds instructions rather than revealing existing ones.

What each choice would do here

A · Why this does not meet the full requirement

/doctor, which checks the installation and reports problems with the Claude Code setup

Checks installation health, not instruction files.

/doctor diagnoses installation and configuration health, such as version and setup issues. It is useful when Claude Code misbehaves at a system level, not for tracing which instruction file a behaviour comes from.

B · Why this meets the requirement

/memory, which lists the instruction files loaded for the session and lets you open each one

Lists the loaded instruction files, including those above the repository.

Claude Code loads CLAUDE.md files from the working directory up through its parents, plus user-level memory. /memory lists what was actually loaded and opens any file, so the consultant quickly finds the workspace-level CLAUDE.md with the changelog rule.

C · Why this does not meet the full requirement

/clear, which empties the conversation so the session starts again without inherited instructions

Clears the conversation; the same files load again anyway.

/clear resets the conversation, but instruction files are loaded from disk at session start and again afterwards. The changelog rule would return immediately, and the consultant would learn nothing about its source.

D · Why this does not meet the full requirement

/init, which scans the repository and writes a CLAUDE.md describing the conventions it finds

Generates a new file; it does not reveal which ones are loaded.

/init analyses the codebase and writes a starter CLAUDE.md. That adds another instruction file to the mix and says nothing about the one already imposing the changelog rule, so the mystery gets harder, not easier.

Check your understanding · explain the design in your own words

How do you see which instruction files a session is using?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Which instructions are loaded Study: CLAUDE.md: the memory hierarchy

Study: Which instructions are loaded Study: CLAUDE.md: the memory hierarchy CLAUDE.md Claude Code

Return to Q262's situation and clause-by-clause explanation

Q263. A design-system repository keeps Storybook story files next to their components, as Card.stories.tsx beside Card.tsx, across about sixty folders at different depths. The team wants its story-writing conventions applied only when Claude works on story files. Which setup does that? A) An import line in the root CLAUDE.md that pulls in a separate story-conventions file at startup B) A CLAUDE.md inside src/ with the story conventions, so they load whenever Claude works under that folder C) A rules file whose paths list is ["**/.stories.tsx"], so it applies to story files at any depth D) A rules file whose paths list is ["src/.stories.tsx"], so it applies to story files under the source folder

Understand Q263 · the situation and each part of the question

The real situation behind this question

A company maintains a shared component library with a story file next to each component so designers can preview it. Story files have their own conventions: argument naming, how controls are declared, which decorators to use. Those rules are irrelevant and distracting when Claude edits the components themselves. The folders are nested to different depths because components are grouped by family. The team wants the story rules to appear only when a story file is being worked on, wherever it lives.

Read the question in parts

Part 1 · across about sixty folders at different depths

The pattern must match at any nesting level, which requires a recursive glob rather than a single folder.

Part 2 · only when Claude works on story files

The conventions must load conditionally by file type, not for a whole folder or every session.

What you are being asked to decide

Which glob scopes rules to one file type spread across many folders? Write the glob that matches a cross-cutting file type.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
globa file pattern; ** matches any number of folders
co-locatedstory files live in the same folder as the component they describe

Study: .claude/rules: path-scoped conventions frontmatter yaml CLAUDE.md

Understand Q263's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When a file type is scattered through the tree, scope by pattern, not by folder. A rules file whose paths use a recursive glob such as **/*.stories.tsx loads for every matching file at any depth and for nothing else.

Connect the design to the situation

Story files sit beside components in sixty nested folders.

Best answer: C. The recursive pattern **/*.stories.tsx matches story files in every folder at every depth, and a rules file with that paths list loads only when Claude works with matching files. The single-star pattern only matches files directly inside src/, missing almost all of the sixty folders. A CLAUDE.md in src/ loads for every file there, components included.

What each choice would do here

A · Why this does not meet the full requirement

An import line in the root CLAUDE.md that pulls in a separate story-conventions file at startup

Imports load at startup for every session, so nothing is scoped.

Imported files in CLAUDE.md are expanded when the session starts, so this is equivalent to putting the conventions in the root file. They would be present for every task, story-related or not.

B · Why this does not meet the full requirement

A CLAUDE.md inside src/ with the story conventions, so they load whenever Claude works under that folder

Loads for every file under src/, not only story files.

A CLAUDE.md in src/ is loaded when Claude works with files under src/, which includes every component file. The story conventions would then show up during component edits, which is exactly the noise the team wants to avoid.

C · Why this meets the requirement

A rules file whose paths list is ["**/*.stories.tsx"], so it applies to story files at any depth

Recursive glob plus path scoping matches every story file and nothing else.

The double star crosses any number of folders, so every Card.stories.tsx and Button.stories.tsx matches wherever it sits. Because the list is in paths frontmatter, the conventions load only when Claude reads or edits one of those files.

D · Why this does not meet the full requirement

A rules file whose paths list is ["src/*.stories.tsx"], so it applies to story files under the source folder

A single star does not cross folders, so nested story files are missed.

In a glob, a single star matches within one path segment, so src/*.stories.tsx only matches story files sitting directly in src/. Components nested in family folders would never trigger the rule, and the team would think scoping was broken.

Check your understanding · explain the design in your own words

Which glob scopes rules to one file type spread across many folders?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: .claude/rules: path-scoped conventions

Study: .claude/rules: path-scoped conventions frontmatter yaml CLAUDE.md

Return to Q263's situation and clause-by-clause explanation

Q264. A team lead wrote the team's commit-message convention into her own ~/.claude/CLAUDE.md, and her sessions follow it perfectly. The CI job that runs claude -p on a build runner to prepare release commits ignores the convention entirely, and so do her teammates' sessions. What explains this? A) Print mode skips every instruction file, so headless runs never see conventions from any CLAUDE.md B) User-level instructions apply only after they are imported from the project CLAUDE.md of each repository C) User-level instructions load only in interactive sessions that are started by the same account that created the file D) The file lives in her home directory on her machine, so the runner and teammates never have it; it belongs in the committed project CLAUDE.md

Understand Q264 · the situation and each part of the question

The real situation behind this question

A team lead cares about clean history and wrote a detailed commit-message convention into her personal Claude Code instructions. Her own sessions produce perfect messages. The team also has a CI job on a build runner that uses Claude Code headless to prepare release commits, and those messages ignore the convention. Teammates' sessions ignore it too. She assumed the instructions were attached to the project because she wrote them while working in it, and now wants to understand why only she benefits.

Read the question in parts

Part 1 · into her own ~/.claude/CLAUDE.md

The instruction is stored in a personal location that exists only in her home directory.

Part 2 · on a build runner

CI runs on a different machine with only the repository checkout, so personal files never appear there.

What you are being asked to decide

Why does a convention work for one person but not for CI or teammates? Match instruction scope to where the file lives and who can see it.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
user-level CLAUDE.mdpersonal instructions in one user's home directory, applied to all their projects
build runnera CI machine that checks out the repository and runs Claude Code headless

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded CLAUDE.md headless

Understand Q264's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Instruction files apply where they physically exist. A personal ~/.claude/CLAUDE.md never leaves one machine, so a team convention must live in the committed project CLAUDE.md to reach teammates and CI.

Connect the design to the situation

A commit convention in a personal file never reaches the build runner.

Best answer: D. ~/.claude/CLAUDE.md is a personal file on one machine; nothing copies it to a build runner or to other developers. A CLAUDE.md committed in the repository travels with every clone, including the one CI checks out. Headless runs do load instruction files, so print mode is not the cause.

What each choice would do here

A · Why this does not meet the full requirement

Print mode skips every instruction file, so headless runs never see conventions from any CLAUDE.md

Headless runs load instruction files the same way interactive ones do.

Print mode loads project and user instruction files just like an interactive session. If the runner had the file, it would apply. The difference is that the runner's account has no such file at all.

B · Why this does not meet the full requirement

User-level instructions apply only after they are imported from the project CLAUDE.md of each repository

User-level files load on their own for that user; imports are not required.

A user-level CLAUDE.md is loaded automatically for every project that user opens. No import is needed, which is why the lead's own sessions work without any change to the repository.

C · Why this does not meet the full requirement

User-level instructions load only in interactive sessions that are started by the same account that created the file

Loading does not depend on interactivity; the file simply is not on those machines.

Instruction loading does not depend on interactivity or on which account wrote the file. Each machine loads the user file of whoever runs Claude Code there, and on the runner that file does not contain her convention.

D · Why this meets the requirement

The file lives in her home directory on her machine, so the runner and teammates never have it; it belongs in the committed project CLAUDE.md

Only the committed project file reaches other machines and people.

The file sits in her home directory, outside any repository, so git never carries it. Teammates and the runner therefore never load it. Moving the convention into the committed project CLAUDE.md puts it into every clone and every CI checkout.

Check your understanding · explain the design in your own words

Why does a convention work for one person but not for CI or teammates?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded CLAUDE.md headless

Return to Q264's situation and clause-by-clause explanation

Q265. A dependency-audit skill lists every transitive package, its licence and its known advisories, often thousands of lines. After running it, developers find Claude drifting toward the audit instead of the feature they were building. The team adds context: fork to the skill's frontmatter. Which statement about that change is exactly right? A) It runs the skill in a separate subagent context and returns only the final result to the main conversation B) It switches the skill to a smaller model, which produces shorter output and therefore less clutter in the session C) It stops Claude from invoking the skill automatically, so the audit only runs when a user types its name D) It copies the whole current conversation into the skill so the audit can use everything discussed so far

Understand Q265 · the situation and each part of the question

The real situation behind this question

A security-minded team built a skill that audits dependencies: every transitive package, its licence and any published advisories. The output is thorough and often runs to thousands of lines. Developers run it mid-task, and afterwards Claude keeps returning to licence questions instead of the feature at hand, because the audit now dominates the conversation. Someone adds the fork setting to the skill's frontmatter, and the team wants to be sure what that setting does and does not change.

Read the question in parts

Part 1 · often thousands of lines

The skill's intermediate output is large enough to crowd the working conversation.

Part 2 · Which statement about that change is exactly right?

The item tests the precise behaviour of the fork setting, not a general way to reduce noise.

What you are being asked to decide

What does context: fork do to a skill? Know the exact effect of the fork setting versus other frontmatter fields.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
context: forkfrontmatter that runs the skill in its own subagent context
subagenta helper with a separate context that returns a result to the caller

Study: Skill frontmatter options Study: Skills vs slash commands vs subagents skill frontmatter context isolation

Understand Q265's answer · full reasoning and worked examples
Claude synthesized · the point, straight

context: fork changes where a skill runs, not who can call it or which model it uses. The skill executes in its own subagent context without the conversation history, and only its result returns, so verbose work cannot derail the main task.

Connect the design to the situation

A noisy audit skill derails the developer's main task.

Best answer: A. context: fork runs the skill as a subagent in its own context, with the skill content as its task, so the thousands of lines of intermediate output stay there and only the result comes back. The forked run does not receive the conversation history. Preventing automatic invocation is a different frontmatter field.

What each choice would do here

A · Why this meets the requirement

It runs the skill in a separate subagent context and returns only the final result to the main conversation

Isolated run, result only returned: exactly what fork does.

With context: fork the skill runs in its own subagent context, optionally with a chosen agent type. The package-by-package output stays in that context, and the main conversation receives only the summary, so the developer's task stays in focus.

B · Why this does not meet the full requirement

It switches the skill to a smaller model, which produces shorter output and therefore less clutter in the session

Model choice is a separate field; fork changes where the skill runs.

A skill can set a different model in its frontmatter, but that is a separate field and does not isolate anything. A smaller model would still pour its output into the main conversation.

C · Why this does not meet the full requirement

It stops Claude from invoking the skill automatically, so the audit only runs when a user types its name

That is the job of disable-model-invocation, not context: fork.

Restricting invocation to users is controlled by disable-model-invocation. The fork setting does not change who can trigger the skill; it changes where the work happens once it is triggered.

D · Why this does not meet the full requirement

It copies the whole current conversation into the skill so the audit can use everything discussed so far

A forked skill starts without the conversation history; it gets the skill content as its task.

This reverses the real behaviour. A forked skill runs as a subagent whose task is the skill content; it does not inherit the conversation history. If the audit needs context from the conversation, the arguments passed to the skill must carry it.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Skill frontmatter with a forked context

TEXT
---
name: dependency-audit
description: Audit transitive dependencies for licence and advisory issues.
context: fork
---
List every transitive dependency with licence and known advisories.
Return only: packages needing action, why, and the suggested fix.

The full scan happens in the forked context; the instructions also ask for a compact result, which is what the main conversation receives.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What does context: fork do to a skill?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Skill frontmatter options Study: Skills vs slash commands vs subagents

Study: Skill frontmatter options Study: Skills vs slash commands vs subagents skill frontmatter context isolation

Return to Q265's situation and clause-by-clause explanation

Q266. A developer must add account deletion to a SaaS product. Legal rules about what must be kept for audit, how backups are handled, and how long invoices are retained are not written down anywhere; only the developer and a lawyer know them. Before any code is written, what is the best way to bring the unstated constraints into the design? A) Ask Claude to scan the existing deletion jobs and infer the legal retention rules from how the code behaves today B) Start with a failing end-to-end deletion test and let each failure reveal which data and systems are involved C) Ask Claude to interview the developer with targeted questions about retention, backups and audit needs, then write a spec from the answers D) Ask Claude to compare three open-source account-deletion libraries and recommend the one with the most features

Understand Q266 · the situation and each part of the question

The real situation behind this question

A small SaaS company has to ship account deletion to satisfy a privacy regulation. Deleting a row is easy; deciding what must survive is not. Invoices have to be kept for years, audit logs have their own rules, and backups raise questions nobody has written down. The developer has notes from a meeting with the company lawyer, and some of it is only in memory. The developer wants Claude Code to help, but a design that ignores one rule would mean costly rework or a compliance problem.

Read the question in parts

Part 1 · are not written down anywhere

No file or document holds the constraints, so reading the codebase cannot surface them.

Part 2 · Before any code is written

The constraints must shape the design first, which rules out discovery through implementation.

What you are being asked to decide

How do you get requirements that only you know into Claude's plan? Choose the interview pattern when constraints are undocumented.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
interview patternClaude asks the developer structured questions before proposing a design
requirement gatheringcollecting constraints and rules before designing a solution

Study: Plan mode vs direct execution and the Explore subagent elicitation requirement-gathering Claude Code

Understand Q266's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When the constraints exist only in people's heads, no amount of reading code or libraries will find them. Having Claude interview the developer and write the answers into a spec brings those constraints into the design before any code depends on them.

Connect the design to the situation

Legal retention rules for account deletion exist only in conversation.

Best answer: C. The missing information is not in the code or in any library; it is in people's heads, so the fastest path is for Claude to ask focused questions and turn the answers into a spec before planning. Inferring rules from existing jobs only reproduces what the code already does, which may be exactly what the legal rules contradict.

What each choice would do here

A · Why this does not meet the full requirement

Ask Claude to scan the existing deletion jobs and infer the legal retention rules from how the code behaves today

Code shows current behaviour, not the legal rules it may already violate.

Reading current deletion jobs shows how the system behaves now, which is useful context. It cannot reveal a rule that exists only in a lawyer's advice, and if the current code is non-compliant, inferring from it bakes the problem in.

B · Why this does not meet the full requirement

Start with a failing end-to-end deletion test and let each failure reveal which data and systems are involved

Tests reveal technical wiring, not unwritten legal obligations.

A failing test is a good way to find which tables and services are touched by deletion. It says nothing about which data must legally be kept, so the design could pass every test and still break retention obligations.

C · Why this meets the requirement

Ask Claude to interview the developer with targeted questions about retention, backups and audit needs, then write a spec from the answers

The knowledge is in people, so asking them surfaces it before design.

Claude asks focused questions, such as which records must be retained, for how long, what happens to backups, and who can restore, and the developer answers from the legal notes. The answers become a written spec that the plan and later reviews can rely on.

D · Why this does not meet the full requirement

Ask Claude to compare three open-source account-deletion libraries and recommend the one with the most features

A library comparison cannot know this company's retention obligations.

Comparing libraries is a reasonable later step once requirements are clear. Feature counts do not encode this company's retention periods or audit needs, so the choice would be made before the real constraints are known.

Check your understanding · explain the design in your own words

How do you get requirements that only you know into Claude's plan?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Plan mode vs direct execution and the Explore subagent

Study: Plan mode vs direct execution and the Explore subagent elicitation requirement-gathering Claude Code

Return to Q266's situation and clause-by-clause explanation

Q267. The team replaced an old logging helper and updated the project CLAUDE.md last week. Everyone's sessions now use the new helper except one developer's, which still writes calls to the old helper. That developer has pulled the latest main. The lead suspects a personal or local instruction file still describes the old helper. Which step confirms or rules out that suspicion before anything is edited? A) Make the new-helper instruction in the project CLAUDE.md more emphatic and see whether the behaviour changes B) Have the developer delete the whole ~/.claude directory so that only the project configuration remains C) Ask Claude in that session which logging helper it should use, and compare its answer with the project file D) Run /memory in that developer's session and check each loaded file for the outdated logging instruction

Understand Q267 · the situation and each part of the question

The real situation behind this question

A payments team retired an old logging helper and replaced it with a structured one. The project instructions were updated and merged, and most developers saw Claude switch immediately. One developer's sessions keep writing calls to the old helper, which reviewers then reject. The developer insists they are on the latest code. The lead remembers that this developer used to keep personal notes for Claude and suspects one of them still describes the retired helper, but wants proof before anyone edits or deletes files.

Read the question in parts

Part 1 · That developer has pulled the latest main

The committed project file is current, so the difference must come from files outside it.

Part 2 · before anything is edited

The step must be diagnostic and non-destructive, confirming the cause first.

What you are being asked to decide

How do you check whether an old instruction file is overriding a new team rule? Diagnose from loaded files before changing configuration.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CLAUDE.local.mda personal, uncommitted instruction file in the project that only one developer has
loaded filesthe instruction files a session actually read at startup

Study: Which instructions are loaded Study: Read the filenames before applying a hierarchy CLAUDE.md Claude Code

Understand Q267's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When one session behaves differently, look at what that session loaded before changing anything. /memory shows every instruction file in play, so a stale personal or local file is found directly instead of being guessed at.

Connect the design to the situation

One developer's sessions still use a retired logging helper.

Best answer: D. /memory lists every instruction file that session loaded, such as user-level CLAUDE.md, CLAUDE.local.md or a parent-folder file, and opens them, so the lead can find the stale text directly. Asking Claude to recite a convention shows what it currently believes, not which file taught it, and deleting ~/.claude destroys settings and history without confirming anything.

What each choice would do here

A · Why this does not meet the full requirement

Make the new-helper instruction in the project CLAUDE.md more emphatic and see whether the behaviour changes

Changes configuration before diagnosing, and may not beat a conflicting file.

Louder wording in the project file might win sometimes, but it changes configuration before the cause is known. If a personal file conflicts, the two instructions keep competing and the behaviour stays inconsistent.

B · Why this does not meet the full requirement

Have the developer delete the whole ~/.claude directory so that only the project configuration remains

Destructive and still does not identify the cause.

Deleting ~/.claude removes personal settings, credentials and session history, and it would not touch a CLAUDE.local.md inside the project. It is destructive, possibly ineffective, and still leaves the lead without an explanation.

C · Why this does not meet the full requirement

Ask Claude in that session which logging helper it should use, and compare its answer with the project file

Shows the belief, not the file it came from.

Claude will report whatever it currently weighs most, perhaps the old helper. That confirms the symptom but not the source: the stale text could be in a user file, a local file or a parent directory, and the answer does not say which.

D · Why this meets the requirement

Run /memory in that developer's session and check each loaded file for the outdated logging instruction

Lists the actual loaded files so the stale one can be found.

/memory shows exactly which memory files the session loaded and lets the lead open each one. If a CLAUDE.local.md or user-level file still describes the old helper, it will be visible, and the fix becomes a targeted edit of that one file.

Check your understanding · explain the design in your own words

How do you check whether an old instruction file is overriding a new team rule?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Which instructions are loaded Study: Read the filenames before applying a hierarchy

Study: Which instructions are loaded Study: Read the filenames before applying a hierarchy CLAUDE.md Claude Code

Return to Q267's situation and clause-by-clause explanation

Q268. An incident-response team writes postmortems with twelve required sections. Engineers paste their own saved versions of a long drafting prompt into Claude Code, and the versions have drifted so some drafts miss sections. The team wants one versioned procedure anyone can run on demand with an incident ID. What should they build? A) Give each engineer a shell alias that runs Claude Code headless with the drafting prompt from a local text file B) Put the full drafting prompt in the root CLAUDE.md so it is present in every session for every engineer C) Add a rules file scoped to postmortems/** so the drafting procedure appears whenever those files are opened D) Create a committed project skill such as .claude/skills/postmortem/SKILL.md with an argument hint for the incident ID

Understand Q268 · the situation and each part of the question

The real situation behind this question

An incident-response team at a logistics company writes a postmortem after every major outage. Their template has twelve sections, from timeline to follow-up owners. Each engineer keeps a personal copy of a long drafting prompt and pastes it into Claude Code. Over months the copies diverged, and recent drafts have missed the customer-impact and follow-up sections. The team lead wants one procedure, reviewed like code, that anyone can start at the end of an incident by giving the incident ID.

Read the question in parts

Part 1 · the versions have drifted

The procedure must have a single source of truth under version control, not personal copies.

Part 2 · anyone can run on demand with an incident ID

The mechanism must be explicitly invoked and accept an argument, which is what a skill or command provides.

What you are being asked to decide

How do you give a team one repeatable prompt they can run whenever needed? Pick the packaging for an on-demand team workflow.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
project skilla committed, named procedure anyone on the team can invoke
argument hintfrontmatter that shows what input the command expects, such as an incident ID

Study: Skills vs slash commands vs subagents skill slash command agent-skills

Understand Q268's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A procedure people run on demand needs a name, an input and one reviewed source. A committed project skill provides all three, while always-loaded instructions, path-triggered rules and personal aliases each miss at least one.

Connect the design to the situation

Drifting copies of a postmortem prompt produce incomplete drafts.

Best answer: D. A committed project skill is invoked by name, takes the incident ID as an argument, and gives every engineer the same reviewed procedure. Putting the prompt in CLAUDE.md shares it but loads it into every unrelated session, and a scoped rules file only appears when postmortem files are touched, which is not the same as running a procedure on demand.

What each choice would do here

A · Why this does not meet the full requirement

Give each engineer a shell alias that runs Claude Code headless with the drafting prompt from a local text file

Per-machine copies drift again and are not reviewed with the code.

Aliases make the prompt quick to run, but each engineer still keeps a local copy. The copies will drift the same way, and changes are never reviewed alongside the repository.

B · Why this does not meet the full requirement

Put the full drafting prompt in the root CLAUDE.md so it is present in every session for every engineer

Shared, but always loaded and never explicitly invoked.

Everyone would get the same text, fixing the drift. But the long prompt would sit in context during every unrelated session, and nothing marks when the procedure should run or which incident it applies to.

C · Why this does not meet the full requirement

Add a rules file scoped to postmortems/** so the drafting procedure appears whenever those files are opened

Triggers on file paths, not on demand, and only as guidance.

A scoped rules file loads guidance when Claude works on files under postmortems/. That helps with formatting existing drafts, but it cannot be run with an incident ID and does nothing until a file already exists.

D · Why this meets the requirement

Create a committed project skill such as .claude/skills/postmortem/SKILL.md with an argument hint for the incident ID

One versioned procedure, run by name, with the incident ID as input.

A SKILL.md committed under .claude/skills/postmortem/ becomes a slash command that every clone has. Its frontmatter can describe the expected incident ID, and the body holds the reviewed twelve-section procedure, so each run follows the same steps.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

A committed postmortem skill

TEXT
---
name: postmortem
description: Draft a postmortem with all twelve required sections for an incident.
argument-hint: <incident-id>
---
Draft the postmortem for incident $ARGUMENTS.
Include every section: summary, impact, customer impact, timeline, detection,
root cause, contributing factors, what went well, what went poorly, actions,
owners, follow-up dates. Mark any section with missing data as TODO.

Invoked as /postmortem INC-4821. The file is reviewed and versioned with the repository, so every engineer runs the same procedure.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you give a team one repeatable prompt they can run whenever needed?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Skills vs slash commands vs subagents

Study: Skills vs slash commands vs subagents skill slash command agent-skills

Return to Q268's situation and clause-by-clause explanation

Q269. A developer has four tasks queued for Claude Code this afternoon. Which one most warrants starting in plan mode rather than asking for the change directly? A) Bump one library by a patch version exactly as a published security advisory instructs B) Move app state from a global store to a server-cache library across seventy components, with several possible migration orders C) Correct a misspelled word in one user-facing error message that a support ticket quoted exactly D) Rename a confusingly named local variable inside a single function that already has full test coverage

Understand Q269 · the situation and each part of the question

The real situation behind this question

A front-end developer at a travel-booking company has a to-do list for the afternoon. Three items are small: a typo a customer reported, a variable name that confuses readers, and a patch upgrade required by a security notice. The fourth is a long-discussed migration away from a global state store toward a server-cache library, touching around seventy components. Several migration orders are possible, some safer than others. The developer wants to use Claude Code efficiently and avoid rework on the one task that can go badly wrong.

Read the question in parts

Part 1 · most warrants starting in plan mode

The question asks which task benefits from exploring and planning before any edits are made.

Part 2 · with several possible migration orders

Multiple valid approaches signal a design decision that should be made deliberately first.

What you are being asked to decide

Which kind of task should start with a plan instead of edits? Match task shape to plan mode versus direct execution.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
plan modea read-only mode where Claude explores and proposes a plan before editing
decompositionbreaking a large change into ordered steps before starting

Study: Plan mode vs direct execution and the Explore subagent Claude Code decomposition

Understand Q269's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Plan mode earns its cost when a task is broad, ambiguous and has several reasonable approaches. Small, precisely described changes go straight to execution; a many-file migration with competing strategies should be explored and planned first.

Connect the design to the situation

A seventy-component state migration versus three one-line fixes.

Best answer: B. The state migration touches many files, has several valid strategies, and needs dependencies mapped before any edit, which is what plan mode is for: read-only exploration and a plan to review first. The other three are small, clearly specified changes where planning adds a step without reducing risk.

What each choice would do here

A · Why this does not meet the full requirement

Bump one library by a patch version exactly as a published security advisory instructs

The advisory already states the exact change: direct execution fits.

Security advisories usually state the exact version to move to. Claude can apply it and run the test suite directly. If the upgrade surprised anyone, that would appear in the test run, not in a plan.

B · Why this meets the requirement

Move app state from a global store to a server-cache library across seventy components, with several possible migration orders

Many files, several valid orders, dependencies to map: plan first.

This change spans dozens of files and needs a strategy: which components first, how to run both systems side by side, how to test each step. Plan mode lets Claude explore read-only and propose an order the developer approves before any file changes.

C · Why this does not meet the full requirement

Correct a misspelled word in one user-facing error message that a support ticket quoted exactly

Exact, one-line and fully specified: direct execution fits.

The fix is a single known string in a single file. Plan mode would make Claude explore and present a plan for a change the developer can already describe exactly, adding a review step with nothing to decide.

D · Why this does not meet the full requirement

Rename a confusingly named local variable inside a single function that already has full test coverage

Scoped to one function with tests: direct execution fits.

A local rename inside one function, protected by tests, is small and low risk. Asking directly lets Claude make the edit and run the tests; planning would not surface anything the tests would not.

Check your understanding · explain the design in your own words

Which kind of task should start with a plan instead of edits?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Plan mode vs direct execution and the Explore subagent

Study: Plan mode vs direct execution and the Explore subagent Claude Code decomposition

Return to Q269's situation and clause-by-clause explanation

Q270. A monorepo's 700-line CLAUDE.md mixed Go, SQL, Helm and front-end conventions, and Go error-handling advice kept appearing in SQL files. The team split it into four files and added an import line for each at the top of the root CLAUDE.md. The bleed continues exactly as before. What is the root cause? A) Imported files are pulled in at launch, so all four sets are in context for every file; conditional loading needs rules files with paths globs B) A user-level instruction file loaded before the project file is repeating the Go advice in every session C) Imported files are loaded only after a matching file is opened, so the conventions arrive too late to shape the edit D) Import lines only resolve absolute paths, so the four files silently fail to load and older cached copies are used

Understand Q270 · the situation and each part of the question

The real situation behind this question

A platform company's monorepo holds Go services, SQL migrations, Helm charts and a React front end. Over two years its CLAUDE.md grew to seven hundred lines. Engineers noticed Claude applying Go error-wrapping advice to SQL files and React naming to Helm values. A developer tidied things by splitting the file into four technology files and importing them from the root CLAUDE.md. The file is much easier to read now, but the cross-technology bleed in Claude's output has not changed at all.

Read the question in parts

Part 1 · added an import line for each at the top of the root CLAUDE.md

The split uses imports, whose loading time decides whether anything became conditional.

Part 2 · The bleed continues exactly as before

The change did not alter what is in context, which is the clue to how imports load.

What you are being asked to decide

Why does splitting CLAUDE.md into imported files not stop rules from bleeding? Tell always-loaded imports apart from path-conditional rules.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
importa line in CLAUDE.md that pulls another file into context at startup
paths globfrontmatter that makes a rules file load only for matching files

Study: .claude/rules: path-scoped conventions Study: CLAUDE.md: the memory hierarchy CLAUDE.md frontmatter context window

Understand Q270's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Splitting CLAUDE.md with imports improves organisation but not scope, because imports are expanded at startup. To keep a technology's conventions out of unrelated edits, put them in rules files with paths globs that load only for matching files.

Connect the design to the situation

Four imported files still put Go advice into SQL edits.

Best answer: A. Imports in CLAUDE.md are expanded when the session starts, so splitting the file changed its organisation but not what is in context: every convention is still present for every edit. Path-scoped rules files with paths frontmatter load only when Claude works on matching files, which is what stops Go advice from reaching SQL edits.

What each choice would do here

A · Why this meets the requirement

Imported files are pulled in at launch, so all four sets are in context for every file; conditional loading needs rules files with paths globs

Imports load up front; only paths-scoped rules load conditionally.

Imported files are expanded into the session when it starts, exactly as if their text were in the root file. Moving each technology into a rules file with paths such as **/*.go or migrations/**/*.sql is what makes each set load only for matching files.

B · Why this does not meet the full requirement

A user-level instruction file loaded before the project file is repeating the Go advice in every session

Nothing suggests a user file; the change the team made explains the result.

A user-level file could add conventions, but nothing here points to one and the Go advice matches the project's own file. The simpler explanation is that the split changed nothing about when content loads.

C · Why this does not meet the full requirement

Imported files are loaded only after a matching file is opened, so the conventions arrive too late to shape the edit

Imports are expanded at launch, not on demand.

This describes path-scoped rules, not imports, and even reversed it would not explain continued bleed. Imports are resolved at launch, so all four sets are present from the first message.

D · Why this does not meet the full requirement

Import lines only resolve absolute paths, so the four files silently fail to load and older cached copies are used

Relative imports work; the files load, which is why the bleed continues.

CLAUDE.md imports accept relative paths, resolved from the importing file. The four files do load, which is exactly why the conventions still appear. There is no cache of older copies involved.

Check your understanding · explain the design in your own words

Why does splitting CLAUDE.md into imported files not stop rules from bleeding?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: .claude/rules: path-scoped conventions Study: CLAUDE.md: the memory hierarchy

Study: .claude/rules: path-scoped conventions Study: CLAUDE.md: the memory hierarchy CLAUDE.md frontmatter context window

Return to Q270's situation and clause-by-clause explanation

Q271. A consultancy uses one repository and Claude Code to produce reports for many clients. A regulated bank needs British spelling, a mandatory risk-disclosure section and a stricter source policy, but only for its own reports; every other client keeps the defaults in the root CLAUDE.md. Analysts start each job from the client's folder. Which configuration fits? A) Add the bank's rules as environment variables in the project settings file so that they are applied to each run B) Create clients/bank/CLAUDE.md with the extra rules, so jobs started in that folder load it on top of the root defaults C) Add an "if the client is the bank" block to the root CLAUDE.md so the model applies the stricter rules when it recognises that client D) Put the bank's rules into each analyst's personal instruction file so that every report they write follows them

Understand Q271 · the situation and each part of the question

The real situation behind this question

A research consultancy produces market reports for dozens of clients from one repository, each client in its own folder. The root instructions define house style, citation format and source standards. A new client, a regulated bank, needs British spelling, a mandatory risk-disclosure section and peer-reviewed sources only, and its compliance team will check every report. The other clients must not change. Analysts already start each job from the relevant client folder, and the partners want the bank's rules to apply automatically without anyone remembering a special step.

Read the question in parts

Part 1 · but only for its own reports

The rules must be scoped to one client's work, not added globally for every job.

Part 2 · Analysts start each job from the client's folder

The launch directory already identifies the client, which directory-level instruction files can use.

What you are being asked to decide

How do you apply special rules for one client without affecting the others? Use directory scoping of instruction files for a subset of work.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
directory-level CLAUDE.mdan instruction file in a subfolder, loaded with its parents when work starts there
defaultsthe root instructions that every client's work inherits

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded CLAUDE.md settings-precedence

Understand Q271's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Instruction files layer by directory: a session loads the CLAUDE.md in its launch folder and every parent. Putting client-specific rules in that client's folder scopes them precisely, where a conditional in the root file leaks them everywhere and relies on the model to notice.

Connect the design to the situation

One bank client needs stricter report rules; other clients keep defaults.

Best answer: B. Claude Code loads CLAUDE.md files from the launch directory and its parents, so a clients/bank/CLAUDE.md adds the bank's rules to the root defaults only for jobs started there. A conditional block in the root file puts the bank's rules into every client's session and relies on the model to recognise when they apply.

What each choice would do here

A · Why this does not meet the full requirement

Add the bank's rules as environment variables in the project settings file so that they are applied to each run

Settings configure the runtime; environment variables are not instructions.

The settings file can set environment variables for the tools Claude runs, but those are not instructions the model reads. The report rules would not reach the model, so the bank's requirements would be silently ignored.

B · Why this meets the requirement

Create clients/bank/CLAUDE.md with the extra rules, so jobs started in that folder load it on top of the root defaults

Scoped by folder, layered on the defaults, and shared through the repository.

When an analyst starts Claude Code in clients/bank, it loads that folder's CLAUDE.md along with the root file, so the bank rules add to the defaults. Jobs started in any other client folder never see them, and the file is committed for every analyst.

C · Why this does not meet the full requirement

Add an "if the client is the bank" block to the root CLAUDE.md so the model applies the stricter rules when it recognises that client

Always loaded for every client and depends on the model spotting the condition.

A conditional block works only if the model notices the condition, and it places the bank's rules in context for every other client too. A slip in either direction produces a wrong report, which is the kind of risk the partners want to remove.

D · Why this does not meet the full requirement

Put the bank's rules into each analyst's personal instruction file so that every report they write follows them

Would apply to every client the analyst serves and is not shared.

Personal instruction files apply to everything that analyst does, so other clients' reports would pick up British spelling and the risk section. They also are not shared, so a new analyst would miss the rules entirely.

Check your understanding · explain the design in your own words

How do you apply special rules for one client without affecting the others?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded

Study: CLAUDE.md: the memory hierarchy Study: Which instructions are loaded CLAUDE.md settings-precedence

Return to Q271's situation and clause-by-clause explanation

Q272. A database team defines three Claude Code subagents: schema-reader should only inspect code, migration-writer creates migration files, and release-runner executes the deploy script. In a trial, schema-reader edited a file while "helpfully" fixing a typo. How should the team make sure each subagent can only use the tools of its role? A) Give all three subagents every tool and have the main agent route tasks so that each role gets suitable work B) Add a deny rule for Edit and Write to the project settings so that no agent in the session can modify files C) Add a sentence to each subagent's system prompt stating which tools it may use and which it must avoid D) Set the tools field in each subagent's definition, for example Read, Grep and Glob only for schema-reader

Understand Q272 · the situation and each part of the question

The real situation behind this question

A database team split their migration workflow across three Claude Code subagents to keep responsibilities clear: one that reads schema and code, one that writes migration files, and one that runs the deploy script. During a trial the schema reader spotted a typo in a comment and fixed it, editing a file nobody expected it to touch. It was harmless, but the team realised that the same freedom could let the reader run the deploy script. They want the roles enforced, not merely described.

Read the question in parts

Part 1 · schema-reader should only inspect code

This role must be read-only, so write and shell tools must be unavailable to it.

Part 2 · can only use the tools of its role

The control must be per subagent and must enforce, which rules out shared or advisory options.

What you are being asked to decide

How do you stop a helper agent from using tools outside its role? Scope capability per subagent in configuration rather than instructions.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
tools fieldthe list in a subagent definition that sets which tools it can call
least privilegeeach role gets only the capabilities its job needs
subagenta specialised helper with its own prompt, context and tool set

Study: Tool availability versus permission Study: Read an allowed-tools list precisely subagent least privilege allowlist

Understand Q272's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Describing a role in a prompt is guidance; limiting its tools is enforcement. Setting the tools list in each subagent's definition gives each role exactly its capabilities, while a shared deny rule is too broad and routing is still a model choice.

Connect the design to the situation

A read-only helper edited a file while fixing a typo.

Best answer: D. The tools field in a subagent's frontmatter defines which tools that subagent can call, so schema-reader simply has no Edit or Write to use. A prompt sentence is guidance the subagent already overrode once, and a project-wide deny rule would also block migration-writer, whose job is to write files.

What each choice would do here

A · Why this does not meet the full requirement

Give all three subagents every tool and have the main agent route tasks so that each role gets suitable work

Routing is a choice made by a model, not a restriction.

Routing work carefully keeps most tasks with the right role, but every subagent could still call every tool. The trial failure was exactly a subagent choosing an extra action inside its assigned task, which routing does not prevent.

B · Why this does not meet the full requirement

Add a deny rule for Edit and Write to the project settings so that no agent in the session can modify files

Blocks the writer role too, since it applies to everyone.

A deny rule in project settings is enforced, which is the right kind of control, but at the wrong scope. It would apply to every agent in the session, so migration-writer could no longer create migration files.

C · Why this does not meet the full requirement

Add a sentence to each subagent's system prompt stating which tools it may use and which it must avoid

Instructions are guidance; the reader already ignored its role once.

A sentence in the subagent prompt describes the role well, and it is worth keeping. But it is guidance, and the trial already showed a helpful model overriding it. It cannot stop the reader from editing or running the deploy script.

D · Why this meets the requirement

Set the tools field in each subagent's definition, for example Read, Grep and Glob only for schema-reader

Each role's tool list is enforced by configuration.

In .claude/agents/schema-reader.md, setting tools to Read, Grep and Glob gives that subagent only those capabilities. migration-writer gets Read and Write, release-runner gets Bash, and each is restricted at the configuration level.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

A read-only subagent definition

TEXT
---
name: schema-reader
description: Inspects schema and data-access code and reports findings. Never modifies files.
tools: Read, Grep, Glob
---
You analyse database schema and queries and report findings with file references.

The tools line is what enforces the role; the description and body explain it. Omitting tools would make the subagent inherit every tool, including Edit and Bash.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you stop a helper agent from using tools outside its role?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Tool availability versus permission Study: Read an allowed-tools list precisely

Study: Tool availability versus permission Study: Read an allowed-tools list precisely subagent least privilege allowlist

Return to Q272's situation and clause-by-clause explanation

Q273. A billing agent built on the Agent SDK can call an issue_refund tool. Finance now requires a named approver to sign off every refund before money moves; the agent should still identify refunds and prepare them. Which change meets the requirement? A) Keep issue_refund but cap each run at one turn so the agent can issue at most a single refund per conversation B) Keep issue_refund and add a system prompt rule saying the agent must never issue a refund without approval C) Keep issue_refund and add a PostToolUse hook that notifies the approver about each refund as soon as it is issued D) Remove issue_refund from the agent and give it a propose_refund tool that only writes a pending request to the approval queue

Understand Q273 · the situation and each part of the question

The real situation behind this question

An online retailer runs a billing agent built on the Agent SDK. It reads customer tickets, checks orders and, until now, issued refunds directly through a tool. After an audit, finance ruled that every refund needs sign-off from a named approver before any money moves. The agent is still valuable for finding eligible cases and preparing the amounts and reasons. The engineering lead has to change the design so the agent keeps doing the analysis while the actual refund can only happen after a person approves it.

Read the question in parts

Part 1 · before money moves

Approval must happen before execution, so any after-the-fact control fails the requirement.

Part 2 · should still identify refunds and prepare them

The agent keeps its analysis role and needs a way to hand off a complete proposal.

What you are being asked to decide

How do you make an agent ask for approval instead of acting? Separate proposing an action from executing it when approval is required.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
human in the loopa person must approve before a consequential action executes
request-only toola tool that records a pending request and performs no restricted action

Study: Request approval without executing the action Study: Where enforcement actually lives HITL least privilege Agent SDK

Understand Q273's answer · full reasoning and worked examples
Claude synthesized · the point, straight

If a human must approve before an action happens, the agent must not hold the ability to perform it. Replace the executing tool with a request-only path, and let a separately authorised executor act after sign-off; notifications and prompt rules do not stop the action.

Connect the design to the situation

A billing agent must propose refunds, not issue them.

Best answer: D. Removing issue_refund means the agent cannot move money at all, and propose_refund records a pending request that a separate, authorised executor completes after the approver signs off. A notification after issuing is too late because the refund already happened, and a prompt rule is guidance the agent can still break.

What each choice would do here

A · Why this does not meet the full requirement

Keep issue_refund but cap each run at one turn so the agent can issue at most a single refund per conversation

Limits volume, not authority; one unapproved refund still happens.

Limiting turns reduces how many actions a run can take but does not remove the authority to refund. A single unapproved refund in the only turn still violates the requirement, and the limit also cripples the agent's analysis.

B · Why this does not meet the full requirement

Keep issue_refund and add a system prompt rule saying the agent must never issue a refund without approval

Guidance only; the capability to refund remains.

A system prompt rule states the policy clearly and will usually be followed. The agent still holds a working refund tool, so a confused or manipulated run can issue a refund anyway, which the finance rule does not allow.

C · Why this does not meet the full requirement

Keep issue_refund and add a PostToolUse hook that notifies the approver about each refund as soon as it is issued

Detective and after the fact: the money has already moved.

A PostToolUse notification runs after issue_refund has completed, so the approver learns about money that has already left. It is a useful audit signal, but it is detective, not preventive, and fails the explicit before-money-moves rule.

D · Why this meets the requirement

Remove issue_refund from the agent and give it a propose_refund tool that only writes a pending request to the approval queue

The agent can only propose; execution waits for approval elsewhere.

Without issue_refund the agent has no way to move money. propose_refund writes a pending record with order, amount and reason; after the approver signs off, a separate service with refund permission executes it, ideally once per request id.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Agent options: the refund capability is gone

PYTHON
from claude_agent_sdk import ClaudeAgentOptions

options = ClaudeAgentOptions(
    mcp_servers={"billing": billing_server},
    allowed_tools=[
        "mcp__billing__lookup_order",
        "mcp__billing__propose_refund",
    ],
    disallowed_tools=["mcp__billing__issue_refund"],
)

The billing server should also stop exposing issue_refund to this agent, and the agent's backend identity should lack refund permission. A separate executor processes approved requests.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you make an agent ask for approval instead of acting?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Request approval without executing the action Study: Where enforcement actually lives

Study: Request approval without executing the action Study: Where enforcement actually lives HITL least privilege Agent SDK

Return to Q273's situation and clause-by-clause explanation

Q274. A custom agent loop built on the Messages API triages support tickets. Its registry currently offers eight tools, including refund_order, close_account and reset_password, and a new tool is added most months. The triage job needs only read_ticket, search_kb and add_label. A security review asks for a configuration that stays safe as the registry grows. Which one? A) Build the request's tools array from an explicit list of read_ticket, search_kb and add_label, so nothing else is ever offered B) Offer every registry tool except refund_order, close_account and reset_password, which are filtered out by name C) Offer every registry tool and log each call to a restricted tool so the security team can review misuse weekly D) Offer every registry tool and state in the system prompt that only the three triage tools may be used

Understand Q274 · the situation and each part of the question

The real situation behind this question

A software company's support team uses a custom agent loop on the Messages API to triage incoming tickets. The application keeps a shared tool registry for several agents, and it now includes powerful tools such as refunds, account closure and password resets. Platform engineers add a tool most months. A security review noticed the triage agent is offered every registered tool even though it only reads tickets, searches the knowledge base and adds labels, and asked for a configuration that stays safe as the registry keeps growing.

Read the question in parts

Part 1 · a new tool is added most months

The design must remain safe for tools that do not exist yet, which favours default exclusion.

Part 2 · stays safe as the registry grows

Blocking only today's dangerous tools is insufficient; future tools must be excluded unless approved.

What you are being asked to decide

How do you limit an agent to the tools it needs even as new tools appear? Choose allowlist over denylist for least-privilege tool exposure.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
allowlistonly listed tools are offered; everything else is excluded by default
tool registrythe application's catalogue of tools that could be offered
least privilegethe agent gets exactly the capabilities its job needs

Study: Tool availability versus permission Study: Read an allowed-tools list precisely allowlist least privilege tool registry

Understand Q274's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Least privilege means naming what an agent may use, not what it may not. An explicit allowlist keeps new tools out by default, while denylists, prompt rules and logs each leave the agent able to call something it should not.

Connect the design to the situation

A triage agent needs three of eight tools, and more are added monthly.

Best answer: A. An explicit allowlist means the model is only offered the three triage tools, and a tool added next month is excluded until someone deliberately adds it. A name-based denylist blocks today's dangerous tools but offers each new tool automatically, which is the growth risk the review raised.

What each choice would do here

A · Why this meets the requirement

Build the request's tools array from an explicit list of read_ticket, search_kb and add_label, so nothing else is ever offered

Only the needed tools are offered, and new ones stay out by default.

The harness builds the tools array from three named entries, so the model can only request those operations. When a new tool is registered next month, triage never sees it unless someone deliberately adds it. Backend credentials for triage should be read-mostly as a second layer.

B · Why this does not meet the full requirement

Offer every registry tool except refund_order, close_account and reset_password, which are filtered out by name

New tools are offered automatically until someone adds them to the denylist.

Filtering out three names blocks today's risky tools. The next tool added, perhaps delete_attachment, would be offered to triage automatically until someone remembers to extend the filter, which is exactly the growth risk raised.

C · Why this does not meet the full requirement

Offer every registry tool and log each call to a restricted tool so the security team can review misuse weekly

Detects misuse after the fact instead of preventing it.

Weekly log review would reveal misuse eventually, which is valuable for audit. But the refund or account closure would already have happened, so it detects damage rather than preventing it.

D · Why this does not meet the full requirement

Offer every registry tool and state in the system prompt that only the three triage tools may be used

Every tool is still offered; the prompt is only guidance.

The model would still see refund, closure and reset tools in its request, and a system prompt rule is guidance. A confused or injected ticket could lead the agent to call a powerful tool that was offered to it.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Build the tools array from an allowlist

PYTHON
TRIAGE_TOOLS = ["read_ticket", "search_kb", "add_label"]

def tools_for_triage(registry):
    return [registry[name] for name in TRIAGE_TOOLS]

response = client.messages.create(
    model=MODEL,
    max_tokens=1024,
    tools=tools_for_triage(REGISTRY),
    messages=messages,
)
🏭 Production shape: registry of decorated tool functions

In production the registry holds @beta_tool functions, not hand-written dicts. The SDK builds each definition from the function. The runner only knows the tools you pass; if the model names any other tool, the runner returns an error result and runs nothing.

PYTHON
import os
from typing import Annotated

import anthropic
from anthropic import beta_tool
from pydantic import Field

from app.tickets import service  # your own ticket service

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-5")


@beta_tool(name="add_label", description="Add one triage label to a ticket.")
def add_label(
    ticket_id: Annotated[str, Field(description="The ticket id, e.g. T-2291.")],
    label: Annotated[str, Field(description="One label from the triage list, e.g. billing.")],
) -> str:
    return service.add_label(ticket_id, label)


# read_ticket, search_kb, close_ticket, issue_refund are defined the same way
REGISTRY = {t.name: t for t in [read_ticket, search_kb, add_label, close_ticket, issue_refund]}
TRIAGE_TOOLS = ["read_ticket", "search_kb", "add_label"]


def triage(messages: list) -> object:
    runner = client.beta.messages.tool_runner(
        model=MODEL, max_tokens=1024,
        tools=[REGISTRY[name] for name in TRIAGE_TOOLS],  # only these exist for this agent
        messages=messages,
        max_iterations=8,
    )
    return runner.until_done()

What the SDK generates:

JSON
{
  "name": "add_label",
  "description": "Add one triage label to a ticket.",
  "input_schema": {
    "type": "object",
    "properties": {
      "ticket_id": {"type": "string", "description": "The ticket id, e.g. T-2291.", "title": "Ticket Id"},
      "label": {"type": "string", "description": "One label from the triage list, e.g. billing.", "title": "Label"}
    },
    "required": ["ticket_id", "label"],
    "additionalProperties": false
  }
}

Tools in app/tools/tickets.py (+ the registry); the triage runner in app/agents/triage.py.

Only named tools are ever offered. The dispatcher should also refuse any tool name outside the list, in case a model output names one anyway.

Not supplied in this example · settings and defaults

context_management — not supplied here. No server compaction strategy is enabled by this body. Host/manual compaction is separate.

output_config — not supplied here. No output_config schema or effort override is supplied. A prompt asking for JSON does not provide schema enforcement.

tool_choice — not supplied here. With tools supplied, the default is auto: the model chooses whether to request a tool. Without tools, no supplied tool can be selected.

system — not supplied here. No application system instructions are supplied through this field. User messages still provide task text.

stop_sequences — not supplied here. No custom stop strings are configured. Generation can still stop for end_turn, tool_use, max_tokens, or another documented reason.

stream — not supplied here. For a direct non-streaming create/parse call, the result is one message rather than an event stream. A stream helper configures this separately.

thinking — not supplied here. No explicit thinking override is shown. Behavior depends on the selected model; omission does not universally mean thinking is off.

cache_control — not supplied here. No top-level automatic cache marker is supplied. Check nested system/tool/content blocks for separate cache_control markers.

metadata — not supplied here. No application user reference is supplied through metadata. Authentication still comes from the request credentials.

betas — not explicitly supplied. No beta list appears in these calls. SDK/client configuration can add headers; inspect those before assuming no beta header is sent.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you limit an agent to the tools it needs even as new tools appear?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Tool availability versus permission Study: Read an allowed-tools list precisely

Study: Tool availability versus permission Study: Read an allowed-tools list precisely allowlist least privilege tool registry

Return to Q274's situation and clause-by-clause explanation

Drill · Prompting & Structured Output (Q275–Q296)

Q275. A hospital network summarizes discharge letters into a structured record. The record schema declares fasting_glucose as a required number, and the extraction runs with strict schema enforcement. Clinicians notice that letters for patients who never had a glucose test still come back with believable values such as 5.4. What is the root cause? A) The schema offers no legal way to express "not stated", so a required number field pushes the model to supply one; allowing null and telling it when to use null removes that pressure. B) Sampling temperature is too high for clinical data; setting it to zero would make the model stop producing values that are not in the letter. C) Strict schema enforcement is not actually active, because a working strict mode would reject any value that does not appear in the discharge letter. D) The discharge letters are too long, so the glucose section falls outside the attention of the model and a larger context window would fix it.

Understand Q275 · the situation and each part of the question

The real situation behind this question

A hospital network wants discharge letters turned into structured records so that diabetes follow-up clinics can find patients with high fasting glucose. The engineering team wrote a tidy schema with every clinical field required and turned on strict enforcement to keep the pipeline from breaking. Weeks later a clinician spots a patient flagged for follow-up who never had blood drawn. The number looked real, it passed every parser, and it was invented. The team needs to understand why a well-typed pipeline is producing believable fiction.

Read the question in parts

Part 1 · declares fasting_glucose as a required number

The schema demands a numeric value on every record, leaving no way to represent a missing test.

Part 2 · never had a glucose test

The source has nothing to extract, so any value in the output must have been produced rather than read.

Part 3 · strict schema enforcement

Generation is constrained to valid structure and types, which says nothing about whether values are grounded in the letter.

What you are being asked to decide

Why does the model invent glucose values for patients who never had the test? Diagnose fabrication caused by schema design rather than by sampling or context size.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Required non-nullable fieldA key that must always be present and must hold a number, so "no value" is impossible to express.
Strict schema enforcementGeneration constrained to the declared shape; it checks structure, never whether a value is true.
TemperatureSampling randomness; lowering it makes wording more repeatable but does not remove a forced value.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Missing values in schemas Structured outputs Temperature

Understand Q275's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Required plus non-nullable means the schema itself demands a value, so when the letter is silent the model must produce one and picks something plausible. Strict mode and temperature act on shape and variability, not grounding. Make the field nullable, tell the model to use null when the value is not stated, and keep grounding checks downstream.

Connect the design to the situation

A discharge letter with no lab section still yields fasting_glucose: 5.4.

Best answer: A. The schema itself demands a number for every letter, so when the letter has none the only schema-valid move is to produce one, and the model picks a plausible value. Strict enforcement guarantees the shape of the output, not that each value came from the source, so the fix belongs in the schema (nullable) plus a clear instruction to return null when the value is absent.

What each choice would do here

A · Why this meets the requirement

The schema offers no legal way to express "not stated", so a required number field pushes the model to supply one; allowing null and telling it when to use null removes that pressure.

Correct: the schema forces a number, so fabrication is the only schema-valid output when the source is silent.

Changing fasting_glucose to accept number or null, and instructing the model to return null when the letter does not report the test, gives the model an honest output for missing evidence. Grounding checks still help, but the structural pressure to invent is gone.

B · Why this does not meet the full requirement

Sampling temperature is too high for clinical data; setting it to zero would make the model stop producing values that are not in the letter.

Temperature changes variability, but even at zero the schema still requires a number that the letter does not contain.

At temperature zero the model picks its single most likely continuation, and with a required number field the most likely continuation is still some typical glucose reading. Randomness was never the cause, so removing it leaves the fabrication in place.

C · Why this does not meet the full requirement

Strict schema enforcement is not actually active, because a working strict mode would reject any value that does not appear in the discharge letter.

Strict mode constrains shape, not truth, so a working strict mode would happily accept an invented number.

Strict mode could be fully active and this would still happen. It guarantees that the output parses and that fasting_glucose is a number; it has no knowledge of the letter, so it cannot reject a plausible but invented value.

D · Why this does not meet the full requirement

The discharge letters are too long, so the glucose section falls outside the attention of the model and a larger context window would fix it.

The problem appears on letters that simply lack the test, which is about missing data rather than attention or window size.

A larger window helps when relevant text is cut off or diluted, but here the letter contains no glucose result at all. Giving the model more room to read does not create data that the document never had.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Schema change that gives absence a legal value

JSON
{
  "type": "object",
  "properties": {
    "fasting_glucose": {
      "type": ["number", "null"],
      "description": "mmol/L exactly as reported. Use null if the letter does not report a fasting glucose result."
    }
  },
  "required": ["fasting_glucose"],
  "additionalProperties": false
}

The key stays required, so every record states an answer, but null is now a valid answer. The description doubles as the instruction for when to use it.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why does the model invent glucose values for patients who never had the test?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Missing values in schemas Structured outputs Temperature

Return to Q275's situation and clause-by-clause explanation

Q276. A ticket-routing service defines exactly one tool, classify_ticket, and leaves tool_choice at its default. About one ticket in forty comes back as a friendly paragraph instead of a tool call, which crashes the router. An engineer changes tool_choice to {"type": "tool", "name": "classify_ticket"}. Which statement about this change is exactly right? A) It makes every response a classify_ticket call instead of prose; with strict: true on the tool the arguments also match its input_schema, but the chosen category can still be wrong. B) It guarantees schema-valid arguments on its own, so the router can skip both strict mode and any validation of the category values it receives. C) It has the same effect as a system prompt line saying "always call classify_ticket", because tool_choice is only a strong hint that the model may still ignore. D) It guarantees the classification values are correct, because a forced tool call makes the model fill the schema from the ticket text rather than guess.

Understand Q276 · the situation and each part of the question

The real situation behind this question

A customer support platform routes incoming tickets by category and urgency. The routing service calls the model with one tool and expects a tool call back every time. Most days that works, but a small fraction of tickets get a warm conversational reply instead, and the router throws an exception and drops the ticket. An engineer flips one setting to force the tool, and the team lead asks what that change does and does not promise before closing the incident.

Read the question in parts

Part 1 · leaves tool_choice at its default

The default is auto, which lets the model decide between calling the tool and replying in text.

Part 2 · comes back as a friendly paragraph instead of a tool call

The failure is the choice of response type, which is what tool_choice controls, not the argument contents.

Part 3 · Which statement about this change is exactly right?

You must pick the statement that neither overstates nor understates the guarantee of forcing the tool.

What you are being asked to decide

What exactly does forcing the one tool fix, and what does it not fix? Separate three guarantees: which operation runs, what shape its arguments take, and whether values are true.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
tool_choiceThe request setting that says whether the model may, must, or must call this specific tool.
strict: trueA tool-definition flag that constrains generated arguments to the input_schema on supported models.
Forced tool callNaming the tool in tool_choice so the response is always that call, never prose.

Study: JSON wording, tool selection, and strict mode are different Study: Structured output — the engineering ladder tool_choice strict Forced tool extraction

Understand Q276's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Three separate knobs: tool_choice picks the operation, strict: true shapes the arguments, and validation checks the values. Forcing the one tool is the precise fix for prose replies, while claims that it also guarantees correct or schema-perfect data overstate it.

Connect the design to the situation

A router that crashes whenever the model replies "Happy to help! This looks like a billing issue."

Best answer: A. Forcing a named tool removes prose as a possible response, which is exactly the failure the router hit. Argument shape is a separate guarantee that comes from strict: true on the tool definition, and neither setting can make the chosen category true, so validation of values still belongs downstream.

What each choice would do here

A · Why this meets the requirement

It makes every response a classify_ticket call instead of prose; with strict: true on the tool the arguments also match its input_schema, but the chosen category can still be wrong.

Correct: it fixes the prose failure, strict adds shape, and value correctness still needs checking.

After the change the API only allows a classify_ticket call, so the router never sees prose again. Adding strict: true to the tool keeps the arguments matching the schema, and the router should still sanity-check categories because both settings are silent about truth.

B · Why this does not meet the full requirement

It guarantees schema-valid arguments on its own, so the router can skip both strict mode and any validation of the category values it receives.

Argument shape comes from strict on the tool; tool_choice alone chooses the operation.

Without strict: true the model usually matches the input_schema but is not constrained to it, so a missing or mistyped field is still possible. Dropping validation on the belief that tool_choice alone guarantees shape would leave the router exposed.

C · Why this does not meet the full requirement

It has the same effect as a system prompt line saying "always call classify_ticket", because tool_choice is only a strong hint that the model may still ignore.

tool_choice is enforced by the API, unlike a prompt instruction that the model can occasionally drift from.

A system prompt line raises the odds of a tool call but cannot rule out text. The tool_choice setting is applied by the API when generating, so the response type is constrained rather than merely requested, which is a stronger guarantee than any instruction.

D · Why this does not meet the full requirement

It guarantees the classification values are correct, because a forced tool call makes the model fill the schema from the ticket text rather than guess.

A forced call guarantees the operation, not the accuracy of what the model puts in it.

The model still reads the ticket and chooses a category; forcing the call only means the choice arrives inside classify_ticket. A misread ticket still gets the wrong category, so the claim of guaranteed correctness is an overstatement that would skip needed checks.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Request with the tool forced and strict arguments

JSON
{
  "model": "claude-model-id",
  "max_tokens": 512,
  "tools": [{
    "name": "classify_ticket",
    "description": "Record the category and urgency of one support ticket.",
    "strict": true,
    "input_schema": {
      "type": "object",
      "properties": {
        "category": {"type": "string", "enum": ["billing", "login", "shipping", "other"]},
        "urgency": {"type": "string", "enum": ["low", "normal", "high"]}
      },
      "required": ["category", "urgency"],
      "additionalProperties": false
    }
  }],
  "tool_choice": {"type": "tool", "name": "classify_ticket"},
  "messages": [{"role": "user", "content": "Ticket: I was charged twice this month."}]
}

tool_choice removes prose as an option; strict keeps category and urgency inside the schema. Whether "billing" is the right category is still a question for your checks.

Not supplied in this example · settings and defaults

context_management — not supplied here. No server compaction strategy is enabled by this body. Host/manual compaction is separate.

output_config — not supplied here. No output_config schema or effort override is supplied. A prompt asking for JSON does not provide schema enforcement.

system — not supplied here. No application system instructions are supplied through this field. User messages still provide task text.

stop_sequences — not supplied here. No custom stop strings are configured. Generation can still stop for end_turn, tool_use, max_tokens, or another documented reason.

stream — not supplied here. For a direct non-streaming create/parse call, the result is one message rather than an event stream. A stream helper configures this separately.

thinking — not supplied here. No explicit thinking override is shown. Behavior depends on the selected model; omission does not universally mean thinking is off.

cache_control — not supplied here. No top-level automatic cache marker is supplied. Check nested system/tool/content blocks for separate cache_control markers.

metadata — not supplied here. No application user reference is supplied through metadata. Authentication still comes from the request credentials.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What exactly does forcing the one tool fix, and what does it not fix?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: JSON wording, tool selection, and strict mode are different Study: Structured output — the engineering ladder

Study: JSON wording, tool selection, and strict mode are different Study: Structured output — the engineering ladder tool_choice strict Forced tool extraction

Return to Q276's situation and clause-by-clause explanation

Q277. A freight forwarder extracts customs data from shipping manifests and retries any record that fails validation, feeding back the error each time. The operations lead wants to stop wasting calls on records that will never succeed. Which failures should be taken out of the retry loop and routed elsewhere? (Select TWO.) A) The tariff code field says "see customs annex", and the annex is a separate file that the pipeline never attaches to the request. B) The manifest template has no consignee phone number anywhere, and the schema still asks for one on every record. C) The response stopped with stop_reason max_tokens halfway through the list of containers, so the JSON never closed. D) The model wrote the gross weight in pounds into the field that expects kilograms, and the validator flagged the implausible value.

Understand Q277 · the situation and each part of the question

The real situation behind this question

A freight forwarder processes thousands of manifests a day and pushes structured customs declarations to a broker system. The pipeline retries failed records up to three times with error feedback, which works well for most problems. But a slice of records fail every single retry and burn calls, delaying the queue. The operations lead wants a rule for which failures get another model call and which go straight to a person or a fetch step.

Read the question in parts

Part 1 · feeding back the error each time

The retries already include feedback, so remaining failures are about missing inputs, not missing guidance.

Part 2 · taken out of the retry loop

You are choosing failures where another model call has no chance of producing a grounded value.

Part 3 · never attaches to the request

The model can only use what is in its context; an unattached file is invisible to every attempt.

What you are being asked to decide

Which validation failures can a retry never fix? Classify failures by whether the needed information is available to the model at all.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Retry with error feedbackResending the task with the specific validation error so the model can correct its output.
Absent source valueData the provided input does not contain, which no amount of re-asking can produce honestly.
max_tokens stopThe response hit its output budget and was cut off before the structure was complete.

Study: Choose the fix — instructions, examples, schema, or checks? Study: What temperature cannot fix Validation and repair Missing values in schemas Stop reason

Understand Q277's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Retries only help when the answer is in front of the model and the first attempt misread or misplaced it. If the value sits in an unattached file or does not exist in the source, every retry is identical and can only fail or fabricate, so route those to fetching, null, or a human.

Connect the design to the situation

A manifest that references an annex the pipeline never sends.

Best answer: AB. When the value lives in a document the model never receives, or does not exist in the manifest at all, every retry sees the same inputs and cannot succeed; those need a human, a fetch of the missing file, or an accepted null. A unit mix-up is exactly what error feedback fixes, and a truncated response is solved by a larger token budget or smaller chunks.

What each choice would do here

A · Why this does not meet the full requirement

The tariff code field says "see customs annex", and the annex is a separate file that the pipeline never attaches to the request.

Correct: the tariff code lives in a file outside the context, so retries see nothing new.

Every retry receives the same manifest with the same pointer to an annex the model cannot open. The only paths to a real value are attaching the annex to the request or sending the record to a person who can look it up, so retries are pure cost.

B · Why this does not meet the full requirement

The manifest template has no consignee phone number anywhere, and the schema still asks for one on every record.

Correct: the value does not exist in the source, so the honest outcome is null or human follow-up.

No manifest from this template has a consignee phone number, so a model that returns one on retry is inventing it. The schema should allow null here and the record should be accepted with null or routed for follow-up, not retried.

C · Why this does not meet the full requirement

The response stopped with stop_reason max_tokens halfway through the list of containers, so the JSON never closed.

Truncation is fixed by raising max_tokens or splitting the work, then retrying.

The model was producing a valid extraction and ran out of output budget. Raising max_tokens, or extracting containers in smaller batches, and retrying resolves it. Removing it from the loop would discard a fixable record.

D · Why this does not meet the full requirement

The model wrote the gross weight in pounds into the field that expects kilograms, and the validator flagged the implausible value.

A unit error is a correctable mistake that a retry with the specific error message usually fixes.

Telling the model that the kilograms field holds a pound value, with the source line quoted, gives it exactly what it needs to convert or re-read. This is the kind of failure the retry loop exists for, and it should stay in it.

Check your understanding · explain the design in your own words

Which validation failures can a retry never fix?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choose the fix — instructions, examples, schema, or checks? Study: What temperature cannot fix

Study: Choose the fix — instructions, examples, schema, or checks? Study: What temperature cannot fix Validation and repair Missing values in schemas Stop reason

Return to Q277's situation and clause-by-clause explanation

Q278. A recruiting platform parses résumés into employment histories. When a record fails validation, the pipeline calls the model again with the same résumé and the same instructions, and the second attempt succeeds only slightly more often than the first. Most failures are end dates written into the start_date field. What is missing from the retry request? A) A higher temperature on the second call, so that the retry explores a different interpretation of the résumé instead of repeating the first one. B) The failed extraction together with the validator message stating which positions have start and end dates reversed, so the model knows what to correct. C) A route to a larger model for every retry, since a model that failed once on a résumé is likely to fail again on the same text. D) Moving the formatting rules from the user message into the system prompt, where the model gives them more weight on the second attempt.

Understand Q278 · the situation and each part of the question

The real situation behind this question

A recruiting platform turns uploaded résumés into structured employment histories that recruiters filter by tenure. A validator checks that each start date comes before its end date and rejects records that fail. The retry step was written quickly: it calls the model again with exactly the same inputs. The team sees almost no lift from retries and is paying for twice the calls. They need to decide what the retry should actually send.

Read the question in parts

Part 1 · with the same résumé and the same instructions

The retry is identical to the first call, so nothing guides the model toward a different answer.

Part 2 · end dates written into the start_date field

This is a field placement error, which is correctable when the model is told exactly where it went wrong.

Part 3 · What is missing from the retry request?

You are identifying the absent ingredient that would make the retry succeed, not a new model or setting.

What you are being asked to decide

Why does the retry barely beat the first try, and what should it contain? Recognize that a retry needs new information, specifically the error, to change the outcome.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Validator messageA precise statement of which field failed which rule, fed back so the model can fix that field.
Retry requestThe follow-up call; it should carry the prior output and the error, not just the original input.
System promptWhere standing instructions live; moving the same rule there adds no information about this failure.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Where every instruction lives in one request Validation and repair tool_result System prompt

Understand Q278's answer · full reasoning and worked examples
Claude synthesized · the point, straight

An identical retry is just another sample of the same distribution. Feed back the failed output and the validator finding that names the reversed positions, and the retry becomes a targeted fix instead of a coin flip.

Connect the design to the situation

A résumé where "2019 to 2022" becomes start_date 2022 and end_date 2019.

Best answer: B. The retry carries no new information, so the model has no reason to answer differently. Including its previous output and the exact validator finding turns the retry into a targeted correction, which is what raises success rates for placement errors.

What each choice would do here

A · Why this does not meet the full requirement

A higher temperature on the second call, so that the retry explores a different interpretation of the résumé instead of repeating the first one.

More randomness may change the output but does not point the model at the specific reversed dates.

Raising temperature produces a different sample, which may or may not fix the reversed dates and may break fields that were correct. It is a lottery rather than a correction, because the model still does not know which positions failed.

B · Why this meets the requirement

The failed extraction together with the validator message stating which positions have start and end dates reversed, so the model knows what to correct.

Correct: the prior output plus the specific error is the new information a retry needs.

The model sees its own previous JSON and a message such as "position 2: start_date 2022 is after end_date 2019; the résumé line reads 2019 to 2022". It can now fix that position deliberately, and the other fields stay stable.

C · Why this does not meet the full requirement

A route to a larger model for every retry, since a model that failed once on a résumé is likely to fail again on the same text.

A different model without the error still has to rediscover the problem on its own.

A larger model might make fewer placement errors in general, but for this record it still starts blind and pays a higher price per call. Adding the error to the request is cheaper and addresses the actual gap.

D · Why this does not meet the full requirement

Moving the formatting rules from the user message into the system prompt, where the model gives them more weight on the second attempt.

The rule is already present; relocating it does not tell the model what it got wrong.

Instructions in the system prompt and in the user turn are both read by the model; placement changes emphasis slightly at most. The failure is not that the rule was unseen, it is that the model was never told which of its outputs broke the rule.

Check your understanding · explain the design in your own words

Why does the retry barely beat the first try, and what should it contain?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Where every instruction lives in one request

Study: Choose the fix — instructions, examples, schema, or checks? Study: Where every instruction lives in one request Validation and repair tool_result System prompt

Return to Q278's situation and clause-by-clause explanation

Q279. An insurer processes claims in two model steps: one assigns the claim to auto, home or travel, and a second extracts the fields that category needs. Travel claims for luggage stolen from a rental home are sometimes routed as home claims, so the second step extracts the wrong fields. A teammate suggests collapsing both steps into one large prompt. Which change most directly addresses the failure? A) Run the routing step three times and use the majority category, so an occasional misroute is outvoted before extraction. B) Send the extraction step to a larger model, so it can notice that the fields it was asked for do not match the claim. C) Keep both steps and add a check between them that tests the assigned category against evidence in the claim, rerouting or flagging it before extraction runs. D) Collapse both steps into one prompt as suggested, since a single call cannot pass a wrong category from one step to the next.

Understand Q279 · the situation and each part of the question

The real situation behind this question

An insurer uses a two-step pipeline to prepare claims for adjusters. The first call decides the line of business and the second pulls the fields that line needs, such as trip dates for travel or property address for home. A recurring edge case, luggage stolen from a vacation rental, lands in the home pipeline. Adjusters then receive property details instead of trip details. A teammate proposes one mega-prompt to avoid handoffs, and the lead must decide.

Read the question in parts

Part 1 · extracts the wrong fields

The visible symptom is downstream, but it is caused by the category passed from the first step.

Part 2 · collapsing both steps into one large prompt

The proposed fix removes the boundary where the intermediate category could be inspected.

Part 3 · most directly addresses the failure

The best answer targets the specific cascading error rather than general quality.

What you are being asked to decide

Where should you fix a misclassification that breaks the next step? Place a validation gate at the handoff between chained prompts instead of merging them.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Prompt chainSeparate model calls where each step feeds the next; errors can cascade across the handoff.
Gate between stepsA check on an intermediate result before the next step uses it.
Focused stepA call that does one job, which is easier to test and fix than a combined prompt.

Study: Focused prompts and pipeline performance Study: Which stages need a model? Prompt chaining Validation and repair Decomposition

Understand Q279's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The error is born in routing and only shows up in extraction. Keep the focused steps, verify the category at the handoff with evidence from the claim, and reroute or flag mismatches before the extractor ever runs.

Connect the design to the situation

A stolen-luggage claim filed from a vacation rental is treated as a home insurance claim.

Best answer: C. The defect is a wrong handoff between focused steps, so the fix belongs at the handoff: verify the category before extraction consumes it. Merging removes the inspection point and makes the prompt do more at once, and majority voting repeats the same systematic confusion rather than catching it.

What each choice would do here

A · Why this does not meet the full requirement

Run the routing step three times and use the majority category, so an occasional misroute is outvoted before extraction.

The confusion is systematic for this claim type, so three runs tend to agree on the same wrong answer.

Voting helps with random noise. Here the model consistently reads rental property wording as home, so three runs usually produce three home votes. You triple the cost of routing without catching the systematic misroute.

B · Why this does not meet the full requirement

Send the extraction step to a larger model, so it can notice that the fields it was asked for do not match the claim.

A bigger extractor still receives the wrong instruction set and has no mandate to reclassify.

The extractor is told to pull home fields and it will try to comply. Even a stronger model is not responsible for second-guessing the category, and the cost grows on every claim while the root cause stays upstream.

C · Why this meets the requirement

Keep both steps and add a check between them that tests the assigned category against evidence in the claim, rerouting or flagging it before extraction runs.

Correct: verifying the category at the handoff stops wrong fields from being extracted.

A cheap check, rules or a short verification call, looks for travel markers such as trip dates, tickets or an away-from-home location. Mismatches get rerouted or flagged, so the extractor only runs with a category the evidence supports.

D · Why this does not meet the full requirement

Collapse both steps into one prompt as suggested, since a single call cannot pass a wrong category from one step to the next.

A merged prompt still has to decide the category, just invisibly, and loses the place to check it.

The single prompt must still decide between home and travel before it knows which fields matter, so the same confusion occurs inside one call. It also mixes two instructions sets in one prompt and removes the point where a check could have caught the category.

Check your understanding · explain the design in your own words

Where should you fix a misclassification that breaks the next step?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Focused prompts and pipeline performance Study: Which stages need a model?

Study: Focused prompts and pipeline performance Study: Which stages need a model? Prompt chaining Validation and repair Decomposition

Return to Q279's situation and clause-by-clause explanation

Q280. A property portal extracts floor area from listings. When the listing has a specification table the area comes back correctly, but when the agent writes it into prose such as "just shy of two thousand square feet over two levels", floor_area returns null. The instructions already define floor_area clearly. What should you try first? A) Make floor_area required and non-nullable, so the model can no longer return null when the area is only described in prose. B) Add a few examples where the area appears in prose, including approximate and spelled-out phrasings, each paired with the correct floor_area value. C) Raise max_tokens so the model has more room to reason through long prose descriptions before it fills in the record. D) Add a regular-expression preprocessor that converts any sentence mentioning square feet into a specification table before the call.

Understand Q280 · the situation and each part of the question

The real situation behind this question

A property portal lets buyers filter homes by size. Its extractor handles listings that include a neat specification table, but many independent agents write descriptions in flowing prose, and those listings come back with no floor area. Buyers filtering by size never see those homes, and agents complain that their listings are invisible. The instructions are already clear about the field, so the team needs the cheapest effective first change.

Read the question in parts

Part 1 · floor_area returns null

The model is not fabricating; it is missing a value that is present in an unfamiliar format.

Part 2 · The instructions already define floor_area clearly

More definition is unlikely to help, so the gap is demonstration of formats rather than meaning.

Part 3 · What should you try first?

The answer should be the lowest-cost, most targeted change before heavier engineering.

What you are being asked to decide

The model finds the area in tables but not in sentences; what do you try first? Pick examples as the lever when the model misses values presented in an unfamiliar format.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Few-shot examplesWorked input and output pairs showing the model how to handle the prose phrasings it misses.
Technique choiceMatching the gap to the tool: definitions for what, examples for how it looks in the wild.
Nullable fieldNull is legitimate for listings without an area, which is why forcing a value would backfire.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Prompting technique choice Missing values in schemas

Understand Q280's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The definition is clear and the value is present, so the model needs to see what prose versions look like. Paired few-shot examples covering those phrasings are the first fix, while forcing non-null values would convert honest misses into fabrications.

Connect the design to the situation

"Roughly 1,950 sq ft" in a description yields floor_area: null.

Best answer: B. The instruction already names the field; what the model lacks is a demonstration that prose phrasings count too. Examples paired with correct values show that pattern directly, while forcing a non-null value would trade misses for invented areas.

What each choice would do here

A · Why this does not meet the full requirement

Make floor_area required and non-nullable, so the model can no longer return null when the area is only described in prose.

Forcing a value on listings that truly lack an area would produce fabricated numbers.

Some listings genuinely omit the area, and a required number would force the model to invent one for them. You would swap visible nulls for invisible wrong numbers, which is worse for a buyer filtering by size.

B · Why this meets the requirement

Add a few examples where the area appears in prose, including approximate and spelled-out phrasings, each paired with the correct floor_area value.

Correct: paired examples teach that prose descriptions contain the value and how to read them.

Three or four examples showing phrases like "just under 2,000 square feet" mapped to 1950 and "about 180 square metres" converted appropriately teach the model that prose counts. Keep null allowed for listings that truly say nothing.

C · Why this does not meet the full requirement

Raise max_tokens so the model has more room to reason through long prose descriptions before it fills in the record.

The output is short; token budget is not what stops the model from recognizing prose phrasings.

The extraction output is a small record and does not come close to the output limit. More budget does not change whether the model recognizes that a sentence contains the area it was asked for.

D · Why this does not meet the full requirement

Add a regular-expression preprocessor that converts any sentence mentioning square feet into a specification table before the call.

Regex rewriting of free prose is brittle and adds a lossy step before the model reads the text.

Agents describe area in countless ways, including approximations and spelled-out numbers. A regex converter will miss many and mangle others, and it moves the recognition problem into brittle code instead of letting the model read language.

Check your understanding · explain the design in your own words

The model finds the area in tables but not in sentences; what do you try first?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples?

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Prompting technique choice Missing values in schemas

Return to Q280's situation and clause-by-clause explanation

Q281. A staffing agency extracts weekly timesheets with tool use and checks each result against a Pydantic model. Every record validates and every field has the right type, yet payroll finds weeks where the seven daily hours do not add up to the weekly_total on the same record. Which statement is exactly right? A) Validation is silently failing, because a model that validates cleanly cannot contain records whose numbers disagree with each other. B) The Pydantic model only checks types, so no rule relates daily hours to weekly_total; add a validator for that sum and feed its error back into a retry. C) Adding ge=0 and le=168 bounds to weekly_total would reject these records, because disagreeing totals fall outside a plausible weekly range. D) Switching the extraction to strict: true would make the daily hours and weekly_total consistent, because strict mode checks the schema more thoroughly.

Understand Q281 · the situation and each part of the question

The real situation behind this question

A staffing agency pays contractors from timesheets that the model extracts from scanned forms. Engineers were proud that every extraction validates against a typed Pydantic model. Payroll then finds several contractors whose daily hours add to 38 while the recorded weekly total is 40, and someone has been paid for the wrong number. The team needs to explain how clean validation let this through and what to change.

Read the question in parts

Part 1 · every field has the right type

Types are satisfied, so the defect must lie in relationships between values rather than in their format.

Part 2 · do not add up to the weekly_total on the same record

This is a cross-field rule, which only an explicitly written validator can check.

Part 3 · Which statement is exactly right?

You must choose the explanation and fix that are both accurate, not merely plausible.

What you are being asked to decide

Why do totals disagree when validation passes, and what fixes it? Distinguish type validation from semantic validation and wire semantic errors into retries.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Semantic validatorA rule relating fields, such as a sum, that type checks cannot express on their own.
strict: trueKeeps tool arguments in the declared shape; it does not check relationships between values.
Schema boundsPer-field limits such as minimum and maximum; they cannot compare one field with another.

Study: Output handling — trust the shape, check the content Study: Choose the fix — instructions, examples, schema, or checks? Validation and repair Strict JSON Schema

Understand Q281's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Validation enforces only the rules you write, and a type model has no rule linking daily hours to the weekly total. Add a semantic validator for the sum, send its specific error into a retry, and flag records where the source form itself disagrees.

Connect the design to the situation

Daily hours sum to 38 but weekly_total says 40, and the record validates.

Best answer: B. Validation only enforces the rules someone wrote; a model of field types cannot notice that two correctly typed fields disagree. Encoding the sum rule as a model validator, with a tolerance and awareness of genuine source discrepancies, turns it into an error message the retry can act on.

What each choice would do here

A · Why this does not meet the full requirement

Validation is silently failing, because a model that validates cleanly cannot contain records whose numbers disagree with each other.

Validation is doing exactly what it was told; the sum rule was never written.

Nothing is failing silently. Pydantic checked every declared type and constraint and they all hold. The mismatch lives in a rule that was never declared, so assuming a library bug would send the team chasing versions instead of writing the rule.

B · Why this meets the requirement

The Pydantic model only checks types, so no rule relates daily hours to weekly_total; add a validator for that sum and feed its error back into a retry.

Correct: write the cross-field rule and use its error to drive a targeted retry.

A model_validator that sums the daily hours and compares them with weekly_total, allowing a small rounding tolerance, raises a precise error. That error goes back to the model in a retry, and if the scanned form itself disagrees, the record is flagged rather than forced to balance.

C · Why this does not meet the full requirement

Adding ge=0 and le=168 bounds to weekly_total would reject these records, because disagreeing totals fall outside a plausible weekly range.

A total of 40 is in range whether or not the days add up to it.

Bounds catch values outside a plausible range, such as 400 hours in a week. A weekly_total of 40 and a daily sum of 38 are each individually plausible, so range limits pass them and the disagreement survives.

D · Why this does not meet the full requirement

Switching the extraction to strict: true would make the daily hours and weekly_total consistent, because strict mode checks the schema more thoroughly.

Strict mode guarantees shape and types, which already pass; it does not reconcile values.

The records already have the right shape and types, which is what strict mode guarantees. It has no notion that one number should equal the sum of seven others, so turning it on changes nothing about these errors.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Cross-field validator that feeds the retry

PYTHON
from pydantic import BaseModel, model_validator

class Timesheet(BaseModel):
    daily_hours: list[float]
    weekly_total: float

    @model_validator(mode="after")
    def totals_agree(self):
        total = round(sum(self.daily_hours), 2)
        if abs(total - self.weekly_total) > 0.05:
            raise ValueError(
                f"daily_hours sum to {total} but weekly_total is {self.weekly_total}; "
                "re-read the form and correct whichever value was misread"
            )
        return self
🏭 Production shape: the validator inside messages.parse

messages.parse sends the schema built from Timesheet and validates the reply with Pydantic, so your cross-field validator runs inside the SDK call. A failure arrives as ValidationError; its text goes into one retry, and a second failure goes to a person.

PYTHON
import os

import anthropic
from pydantic import BaseModel, ValidationError, model_validator

from app.review import review_queue  # your own module

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-5")


class Timesheet(BaseModel):
    daily_hours: list[float]
    weekly_total: float

    @model_validator(mode="after")
    def totals_agree(self):
        total = round(sum(self.daily_hours), 2)
        if abs(total - self.weekly_total) > 0.05:
            raise ValueError(f"daily_hours sum to {total} but weekly_total is {self.weekly_total}; "
                             "re-read the form and correct whichever value was misread")
        return self


def extract(form_text: str, max_attempts: int = 2) -> Timesheet | None:
    messages = [{"role": "user", "content": f"Extract the timesheet.\n<form>{form_text}</form>"}]
    for _ in range(max_attempts):
        try:
            response = client.messages.parse(model=MODEL, max_tokens=1024,
                                             messages=messages, output_format=Timesheet)
            return response.parsed_output
        except ValidationError as err:  # the validator's message names both numbers
            messages = messages[:1] + [{"role": "user", "content": f"Your last answer failed: {err}"}]
    review_queue.put(form_text)  # the form itself may be inconsistent
    return None

Model in app/timesheets/models.py; the extraction call in app/timesheets/extract.py.

The error text names both numbers, so the retry has something concrete to correct. If a retry still disagrees, flag the record for a person, because the scanned form may itself be inconsistent.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why do totals disagree when validation passes, and what fixes it?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Output handling — trust the shape, check the content Study: Choose the fix — instructions, examples, schema, or checks?

Study: Output handling — trust the shape, check the content Study: Choose the fix — instructions, examples, schema, or checks? Validation and repair Strict JSON Schema

Return to Q281's situation and clause-by-clause explanation

Q282. A pharmacy system extracts dosing from prescriptions. Two problems remain: instructions such as "1 tab bid x 10" are read inconsistently, sometimes as ten days and sometimes as ten tablets, and quantity comes back empty on faxed prescriptions where it sits in a margin note. Which few-shot practices fit these problems? (Select TWO.) A) Add a handful of examples of the ambiguous dosing pattern, each with a short note explaining which reading was chosen and why. B) Replace the examples with a longer written definition of every field, since precise definitions remove ambiguity without extra prompt length. C) Include one example for every prescription template the pharmacy has ever received, so that no layout is left unrepresented in the prompt. D) Include examples drawn from different layouts, such as a faxed form with the quantity in a margin note, paired with the correct extraction.

Understand Q282 · the situation and each part of the question

The real situation behind this question

A pharmacy chain uses extraction to pre-fill dispensing records from incoming prescriptions. Pharmacists still check each record, but two recurring errors slow them down: shorthand like "x 10" is interpreted inconsistently, and quantities written in fax margins are missed entirely. The team already uses a couple of generic examples. They want to know what kind of examples to add and how many, without making the prompt unwieldy.

Read the question in parts

Part 1 · read inconsistently

The model has more than one plausible reading and needs to be shown which clue decides it.

Part 2 · quantity comes back empty on faxed prescriptions

The value exists but in a layout the model has not been shown, so layout variety is the fix.

Part 3 · Which few-shot practices fit these problems?

You must pick practices matched to these two specific gaps, not general prompting advice.

What you are being asked to decide

Which ways of using examples fix ambiguous dosing and missed quantities? Apply few-shot examples selectively: targeted for ambiguity, varied for layout coverage.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Targeted examplesA few demonstrations aimed at the specific confusing pattern, with the reason for the chosen reading.
Layout varietyExamples from differently structured documents so the model learns where values can hide.
Prompt budgetEvery example costs context and attention; dozens of routine ones crowd out the important ones.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Prompting technique choice Context budget

Understand Q282's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Examples are most powerful when targeted: a few that show the deciding clue for ambiguous shorthand, and a few from the unusual layouts where values go missing. Exhaustive coverage and longer definitions spend budget without demonstrating the behavior you need.

Connect the design to the situation

"x 10" read as days on one prescription and as tablets on another.

Best answer: AD. Ambiguity is resolved by a few targeted demonstrations that show the deciding clue, and empty fields in unusual layouts are resolved by examples from those layouts. Covering every template bloats the prompt and dilutes the cases that matter, and more prose does not show the model what the margin note looks like.

What each choice would do here

A · Why this does not meet the full requirement

Add a handful of examples of the ambiguous dosing pattern, each with a short note explaining which reading was chosen and why.

Correct: a few targeted examples with the deciding clue resolve the ambiguity.

Three or four examples such as "1 tab bid x 10" with the note "x 10 follows the frequency, so it means days; quantity is 20" show the model which signal to use. Consistency improves because the reasoning, not just the answer, is demonstrated.

B · Why this does not meet the full requirement

Replace the examples with a longer written definition of every field, since precise definitions remove ambiguity without extra prompt length.

Definitions describe fields but do not demonstrate how ambiguous notation should be read.

Definitions tell the model what quantity means, which it already knows. They do not show that a faxed form tucks quantity into a margin note or which reading of "x 10" the pharmacy intends, so behavior changes little.

C · Why this does not meet the full requirement

Include one example for every prescription template the pharmacy has ever received, so that no layout is left unrepresented in the prompt.

Exhaustive template coverage bloats the prompt and buries the ambiguous cases.

Hundreds of templates would consume a large share of the context on routine cases the model already handles. The two real problems would be represented by a sliver of the examples, and cost and latency would rise on every prescription.

D · Why this does not meet the full requirement

Include examples drawn from different layouts, such as a faxed form with the quantity in a margin note, paired with the correct extraction.

Correct: layout variety teaches the model to find quantity in margin notes.

An example of a faxed prescription with the quantity scribbled beside the signature, paired with the correct quantity, teaches the model to look there. It closes the specific gap without adding every other layout.

Check your understanding · explain the design in your own words

Which ways of using examples fix ambiguous dosing and missed quantities?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples?

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Prompting technique choice Context budget

Return to Q282's situation and clause-by-clause explanation

Q283. A podcast network turns episode transcripts into show notes through a write_show_notes tool with tool_choice left on auto, and occasionally receives a written summary instead of a tool call. In the design review, four alternatives are proposed. Which one closes the gap with an actual guarantee? A) Move the sentence "always respond by calling write_show_notes" from the system prompt into the final user message, where it is read last. B) Prefill the assistant turn with an opening brace so the model has to continue a JSON object rather than start a summary. C) Keep the tool and set tool_choice to {"type": "tool", "name": "write_show_notes"}, so the API only allows a call to that tool. D) Drop the tool and ask for JSON in the prompt with a strict format block, since the model evidently prefers writing text for these transcripts.

Understand Q283 · the situation and each part of the question

The real situation behind this question

A podcast network publishes show notes for dozens of episodes a week. A small automation reads each transcript and calls a tool that writes structured notes for the website. Every so often, especially for long rambling episodes, the model writes a friendly prose summary instead, and the publishing job fails. In a design review, four engineers each propose a fix, and the producer wants the one that actually removes the failure rather than making it rarer.

Read the question in parts

Part 1 · tool_choice left on auto

Auto explicitly permits the model to reply in text, which is the exact failure observed.

Part 2 · closes the gap with an actual guarantee

Proposals that only reduce the likelihood of prose do not meet this bar.

Part 3 · four alternatives are proposed

Several options are reasonable techniques; you must find the only enforced control among them.

What you are being asked to decide

Which proposal actually guarantees a tool call instead of prose? Tell an API-enforced control apart from instructions and output nudges.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Forced tool_choiceAn API setting that restricts the response to a call of the named tool.
PrefillStarting the assistant reply yourself; a weak nudge that is unsupported with extended thinking and on the newest models.
Prompt versus schemaAn instruction asks for a format; a schema or tool setting enforces one.

Study: JSON wording, tool selection, and strict mode are different Study: Do not confuse examples with the beginning of an answer tool_choice Prefill Prompt versus schema

Understand Q283's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Auto permits prose, so the fix is to stop permitting it. Forcing the named tool is enforced by the API, while prompt JSON, relocated instructions and prefill all remain requests, and prefill is also unsupported with extended thinking and on the newest models.

Connect the design to the situation

A transcript that yields "Here is a summary of the episode..." instead of show notes.

Best answer: C. Only the tool_choice setting changes what the API permits; the other proposals are requests that make prose less likely without ruling it out. Prompt-only JSON also loses schema enforcement, and prefill is the weakest lever and is not supported with extended thinking or on the newest models.

What each choice would do here

A · Why this does not meet the full requirement

Move the sentence "always respond by calling write_show_notes" from the system prompt into the final user message, where it is read last.

Instruction placement shifts emphasis slightly but cannot forbid a text reply.

The system prompt and the user message are both read in full. Moving the sentence may slightly strengthen it, but it remains a request; the model can still decide that a summary is the helpful response on some transcripts.

B · Why this does not meet the full requirement

Prefill the assistant turn with an opening brace so the model has to continue a JSON object rather than start a summary.

Prefill nudges rather than guarantees and is unavailable in several current configurations.

Starting the reply with a brace makes JSON likely, but the result is free-form text you must parse, and prefill is not supported with extended thinking and is not available on the newest models. It is the weakest structured-output lever available.

C · Why this meets the requirement

Keep the tool and set tool_choice to {"type": "tool", "name": "write_show_notes"}, so the API only allows a call to that tool.

Correct: forcing the named tool is enforced by the API, not requested of the model.

With the named tool forced, the API only accepts a write_show_notes call as the response, so prose is impossible. Add strict: true on the tool if you also need the arguments to match the schema exactly.

D · Why this does not meet the full requirement

Drop the tool and ask for JSON in the prompt with a strict format block, since the model evidently prefers writing text for these transcripts.

Prompt-only JSON can still yield prose or malformed JSON, and it discards the tool schema.

A prompt that asks for JSON is still an instruction; the model can reply with prose or with JSON that is missing fields or badly escaped. You would lose the input_schema and gain nothing enforceable, moving backward from what the tool already provides.

Check your understanding · explain the design in your own words

Which proposal actually guarantees a tool call instead of prose?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: JSON wording, tool selection, and strict mode are different Study: Do not confuse examples with the beginning of an answer

Study: JSON wording, tool selection, and strict mode are different Study: Do not confuse examples with the beginning of an answer tool_choice Prefill Prompt versus schema

Return to Q283's situation and clause-by-clause explanation

Q284. A travel app extracts itineraries from booking confirmation emails through a record_flight tool. Validation often fails because departure_airport and arrival_airport are reversed relative to the "LHR → JFK" route line in the email. Today the retry just resends the email. Which follow-up turn gives the retry the best chance of success? A) A tool_result for the previous call with is_error set, stating that the route line reads LHR → JFK but the fields were recorded as JFK to LHR. B) No retry at all; application code swaps the two airport fields whenever the arrival time on the record is earlier than the departure time. C) A user turn that says only "That extraction was incorrect. Please try again more carefully." and resends the confirmation email. D) Five further identical calls with the same email, keeping the first result that passes validation, since the swap only happens on some attempts.

Understand Q284 · the situation and each part of the question

The real situation behind this question

A travel app reads booking confirmation emails and fills in each user's trip timeline. A validator compares the recorded airports with the route line in the email, and a frequent failure is the two airports being reversed. The retry currently resends the email with nothing else, and it rarely helps. Engineers debate four ways to handle the retry, including a clever code rule that swaps fields automatically.

Read the question in parts

Part 1 · reversed relative to the "LHR → JFK" route line

The validator has concrete evidence of the error, which can be quoted back to the model.

Part 2 · Today the retry just resends the email

The current retry carries no new information, which explains why it rarely succeeds.

Part 3 · Which follow-up turn gives the retry the best chance

You are choosing the content and form of the next message, not a different model or setting.

What you are being asked to decide

What should the retry say so the model fixes the reversed airports? Compose a retry that carries the specific error, in the right message structure.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
tool_result with is_errorThe reply to the model's tool call, marked as an error and carrying the validator's finding.
Specific errorA message naming the fields and the evidence, so the model can correct deliberately.
Heuristic swapCode that rewrites values by rule; it hides the model error and breaks valid edge cases.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Output config during a multi-turn tool loop tool_result Validation and repair Error taxonomy

Understand Q284's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A retry works when it carries new, specific information. Return the failed call with an error that names the reversed fields and quotes the route line, and the model can correct it deliberately, while code swaps and blind repetition either corrupt real cases or rely on luck.

Connect the design to the situation

Departure recorded as JFK and arrival as LHR for a London to New York flight.

Best answer: A. The model needs to know which fields were wrong and what evidence shows it, and returning that as an error result on its own tool call keeps the conversation coherent. A vague retry gives no direction, blind repetition relies on chance, and an automatic code swap would corrupt legitimate cases such as overnight flights crossing time zones.

What each choice would do here

A · Why this meets the requirement

A tool_result for the previous call with is_error set, stating that the route line reads LHR → JFK but the fields were recorded as JFK to LHR.

Correct: a specific error returned on the tool call lets the model fix the swap deliberately.

The model sees its own call answered with an error naming both fields and quoting LHR → JFK. It can issue a corrected record_flight call immediately, and the structured error keeps the tool conversation valid for the next turn.

B · Why this does not meet the full requirement

No retry at all; application code swaps the two airport fields whenever the arrival time on the record is earlier than the departure time.

Arrival can legitimately be earlier in local time, so the swap rule corrupts real itineraries.

An overnight flight from Tokyo to Honolulu arrives before it departs in local time. The rule would swap correct records, and it papers over the model error so nobody learns how often it occurs. Deterministic fixes must be exact, and this one is not.

C · Why this does not meet the full requirement

A user turn that says only "That extraction was incorrect. Please try again more carefully." and resends the confirmation email.

"Try again carefully" gives no information about which fields were wrong.

The model knows something was wrong but not what. It may change a correct field, reformat dates, or return the same swap. Without the evidence from the route line, it is guessing, which is barely better than resending the email.

D · Why this does not meet the full requirement

Five further identical calls with the same email, keeping the first result that passes validation, since the swap only happens on some attempts.

Blind repetition spends calls on chance and does not address the systematic swap.

If the swap happens often, several of the five calls will repeat it, and you pay for all of them. Accepting the first passing result also hides a systematic weakness that a specific error or a targeted example would fix.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Error result that drives the corrected call

JSON
{
  "role": "user",
  "content": [{
    "type": "tool_result",
    "tool_use_id": "toolu_01A",
    "is_error": true,
    "content": "Validation failed: the email route line reads LHR → JFK, but departure_airport=JFK and arrival_airport=LHR. Re-read the route line and call record_flight again with the corrected airports. Other fields passed."
  }]
}

The error answers the model's own tool call, names both fields, quotes the evidence and says which fields to leave alone, so the next call changes only what was wrong.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What should the retry say so the model fixes the reversed airports?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Output config during a multi-turn tool loop

Study: Choose the fix — instructions, examples, schema, or checks? Study: Output config during a multi-turn tool loop tool_result Validation and repair Error taxonomy

Return to Q284's situation and clause-by-clause explanation

Q285. An expense system extracts reports and validates them. On one report the receipts sum to $212.40 but the claimed total reads $221.40. On another, the cost-center box on the form is blank and no cost center appears anywhere else. How should each be handled? A) Send both straight to human review without any retry, since any validation failure on financial data needs a person. B) Retry both, sending the discrepancy for the first and instructing the model to look harder for the cost center on the second. C) Retry the first with the specific discrepancy so the model rechecks receipts and total, flagging it if the source truly disagrees; send the second to review with cost center null. D) For the first, add a "miscellaneous" line of $9.00 so the report balances; for the second, fill in the employee's default cost center.

Understand Q285 · the situation and each part of the question

The real situation behind this question

A mid-size company digitizes employee expense reports and reimburses from the extracted data. Validators check that receipts add up to the claimed total and that every report has a cost center for accounting. Two failures arrive at once: one where the numbers look like a transposed digit, and one where the employee simply left the cost-center box empty. Finance wants a clear, auditable rule for what the pipeline does with each.

Read the question in parts

Part 1 · the receipts sum to $212.40 but the claimed total reads $221.40

The mismatch could be a misread by the model or a genuine error on the form, so it deserves a checked retry.

Part 2 · no cost center appears anywhere else

The value is absent from the source, so no retry can recover it honestly.

Part 3 · How should each be handled?

The two failures have different causes, so the correct answer treats them differently.

What you are being asked to decide

Which failure gets a retry and which goes to a person? Match the remedy to the cause: retry correctable errors, escalate absent information.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Discrepancy retryA retry that quotes the conflicting numbers so the model can recheck what it read.
Human reviewRouting a record to a person when the needed information is not in the source.
Null for absent dataRecording that the value is missing instead of inventing one.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Schema design details that prevent hallucination Validation and repair Escalation Missing values in schemas

Understand Q285's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Diagnose each failure before choosing a remedy. A total mismatch may be a misread, so retry with the specific numbers and flag it if the source truly disagrees; a blank field with no other source is absent data, so record null and route it to a person.

Connect the design to the situation

$212.40 in receipts against a claimed $221.40, and an empty cost-center box.

Best answer: C. The first failure might be a misread digit, which a retry with the exact numbers can fix, and if the form itself disagrees the honest outcome is a flag, not a forced balance. The second value does not exist in the source, so retrying only invites a fabricated cost center.

What each choice would do here

A · Why this does not meet the full requirement

Send both straight to human review without any retry, since any validation failure on financial data needs a person.

The first case is often a cheap-to-fix misread, so skipping the retry wastes reviewer time.

Sending everything to people is safe but expensive, and many discrepancies are simple misreads that a single retry with the numbers resolves in seconds. Human reviewers should see the cases that actually need judgment.

B · Why this does not meet the full requirement

Retry both, sending the discrepancy for the first and instructing the model to look harder for the cost center on the second.

Looking harder cannot find a value that is not on the form and pressures fabrication.

The first half is sound, but telling the model to look harder for a cost center that is not on the form invites it to guess a plausible code. That produces a confident wrong allocation that accounting would never notice.

C · Why this meets the requirement

Retry the first with the specific discrepancy so the model rechecks receipts and total, flagging it if the source truly disagrees; send the second to review with cost center null.

Correct: retry the possible misread with the numbers; escalate the absent cost center.

The retry quotes both figures and asks the model to re-read the receipts and total line. If the extraction was wrong, it gets fixed; if the form really says $221.40, the report is flagged for finance. The blank cost center goes to review as null.

D · Why this does not meet the full requirement

For the first, add a "miscellaneous" line of $9.00 so the report balances; for the second, fill in the employee's default cost center.

Balancing lines and default values invent data and hide real discrepancies from finance.

A made-up balancing line hides a real discrepancy that finance may need to investigate, and a default cost center charges the expense to a department that may not own it. Both changes create data that the source never contained.

Check your understanding · explain the design in your own words

Which failure gets a retry and which goes to a person?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Schema design details that prevent hallucination

Study: Choose the fix — instructions, examples, schema, or checks? Study: Schema design details that prevent hallucination Validation and repair Escalation Missing values in schemas

Return to Q285's situation and clause-by-clause explanation

Q286. A biotech firm extracts fields from 120-page clinical study protocols in one request. Sponsor and phase from the opening pages come back reliably, but dosing schedules and stopping rules buried in later appendices are often missing, and the visit schedule sometimes contradicts the dosing table in the same output. The whole protocol fits comfortably in the context window. What is the best diagnosis and fix? A) Output varies between runs; run the full single-pass extraction three times and keep the most common value for every field. B) Attention is spread across too many fields and pages in one pass; extract per section in focused calls, then run a reconciliation pass that resolves conflicts. C) The protocol is near the window limit and later pages are being truncated; move to a model with a larger context window. D) The model does not realize dosing matters; add capitalized emphasis on dosing and stopping rules at the top of the existing single prompt.

Understand Q286 · the situation and each part of the question

The real situation behind this question

A biotech firm maintains a database of its clinical study protocols for regulatory and operations teams. Each protocol is long, with key operational details in appendices. The extraction runs in one call per protocol. Operations staff keep finding missing dosing schedules and a visit schedule that contradicts the dosing table. A colleague suggests a bigger model, but the documents already fit, and the team needs the right diagnosis.

Read the question in parts

Part 1 · fits comfortably in the context window

Truncation and window size are ruled out, so the cause lies in how the single pass uses the input.

Part 2 · buried in later appendices are often missing

Misses concentrate deep in the document, the signature of attention spread thin over one long pass.

Part 3 · contradicts the dosing table in the same output

Cross-field inconsistency calls for an explicit step that compares and reconciles results.

What you are being asked to decide

Why are late details missed and fields contradictory in one long pass, and what fixes it? Diagnose attention dilution and choose decomposition with reconciliation.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Attention dilutionQuality drops when one pass must track many fields across a very long input.
Reconciliation passA final step that compares per-section results and resolves contradictions.
Context windowHow much input fits; fitting is not the same as being used well.

Study: Focused prompts and pipeline performance Study: PDF reading and processing checkpoints Long-context placement Prompt chaining Context window

Understand Q286's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The document fits, yet late details vanish and fields contradict each other, which is attention dilution. Split the work into focused per-section passes and add a reconciliation pass that compares results and resolves conflicts, instead of buying a bigger window or repeating the flawed pass.

Connect the design to the situation

A dosing table in appendix F is missed while the sponsor name on page one is always found.

Best answer: B. The document fits, so truncation is ruled out; misses concentrated late in a long input, plus internal contradictions, point to one pass doing too much. Focused per-section extraction improves recall, and an explicit reconciliation step is where cross-section conflicts get resolved.

What each choice would do here

A · Why this does not meet the full requirement

Output varies between runs; run the full single-pass extraction three times and keep the most common value for every field.

Voting over the same flawed pass repeats the same systematic misses at triple cost.

If the dosing appendix is systematically underweighted, all three runs tend to miss it. Majority voting also cannot resolve a contradiction that appears in every run, and it triples the cost of an already expensive extraction.

B · Why this meets the requirement

Attention is spread across too many fields and pages in one pass; extract per section in focused calls, then run a reconciliation pass that resolves conflicts.

Correct: focused per-section passes plus reconciliation address both misses and contradictions.

Each call handles one section with only the fields relevant to it, so appendices get full attention. A final pass receives the per-section outputs, detects that the visit schedule and dosing table disagree, and resolves or flags the conflict with source citations.

C · Why this does not meet the full requirement

The protocol is near the window limit and later pages are being truncated; move to a model with a larger context window.

The protocol already fits, so a bigger window does not address the misses.

Truncation would show the last pages missing entirely, but the stem says the protocol fits. A larger window leaves the same single pass trying to track every field across 120 pages, so the misses and contradictions persist at higher cost.

D · Why this does not meet the full requirement

The model does not realize dosing matters; add capitalized emphasis on dosing and stopping rules at the top of the existing single prompt.

Emphasis does not change how thinly one pass spreads across 120 pages and many fields.

Capitalized emphasis may shift priority slightly, but the model still reads one long input while filling many fields. Emphasis tends to cause over-attention to some fields at the cost of others rather than recovering buried details.

Check your understanding · explain the design in your own words

Why are late details missed and fields contradictory in one long pass, and what fixes it?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Focused prompts and pipeline performance Study: PDF reading and processing checkpoints

Study: Focused prompts and pipeline performance Study: PDF reading and processing checkpoints Long-context placement Prompt chaining Context window

Return to Q286's situation and clause-by-clause explanation

Q287. An accessibility review bot comments on front-end pull requests. Teams mute it because it floods reviews with opinions about color palettes, button wording and spacing, burying real problems like missing alt text. The prompt was changed to "flag only significant accessibility issues" and nothing improved. Why did that change fail, and what would work? A) "Significant" has no boundary the model can test; list what to report, such as missing alt text or keyboard traps, and what to skip, such as palette taste. B) The bot should attach a confidence score to each comment and the pipeline should suppress anything the model rates below 0.8. C) The bot needs a hard cap of three comments per pull request, so the most important findings are forced to the top of the list. D) The model is too creative for review work; lowering temperature would make it produce fewer, more consistent comments per pull request.

Understand Q287 · the situation and each part of the question

The real situation behind this question

A product company added an accessibility bot to pull requests to catch problems before release. Instead of helping, it posts a dozen opinions on every change: warmer colors, friendlier labels, more padding. Developers mute it and a missing alt text slips into production. A tech lead edited the prompt to ask for significant issues only, and the noise did not change. The team wants to understand why and fix it properly.

Read the question in parts

Part 1 · flag only significant accessibility issues

This qualifier asks the model to judge significance without defining it, so its judgment stays the same.

Part 2 · burying real problems like missing alt text

The goal is to surface concrete defects, which can be named as explicit report categories.

Part 3 · Why did that change fail, and what would work?

You need both the cause of failure and the replacement technique, not just a knob to turn.

What you are being asked to decide

Why does "only significant issues" not reduce noise, and what does? Replace vague qualifiers with explicit inclusion and exclusion criteria.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Explicit criteriaNamed categories to report and to skip, which give the model a testable rule.
TemperatureControls variation in wording, not which categories count as findings.
CalibrationHow well stated confidence matches real accuracy; self-reported scores are often poorly calibrated.

Study: Prompt iteration as engineering Study: Prompting basics Prompting technique choice Temperature Calibration

Understand Q287's answer · full reasoning and worked examples
Claude synthesized · the point, straight

"Significant" leaves the judgment where it was, so the output does not change. Define the categories to report and the categories to skip, and the model has a concrete filter it can apply to every finding.

Connect the design to the situation

A comment suggesting a warmer shade of blue sits next to an unlabeled icon button.

Best answer: A. The model already believes its palette comments are significant, so a vague qualifier changes nothing. Naming concrete report categories and explicit skip categories gives it a filter it can apply consistently to each finding.

What each choice would do here

A · Why this meets the requirement

"Significant" has no boundary the model can test; list what to report, such as missing alt text or keyboard traps, and what to skip, such as palette taste.

Correct: concrete report and skip categories give the model a boundary it can apply.

A report list such as missing text alternatives, keyboard traps, focus order, contrast below the chosen standard, plus a skip list such as palette taste and copy tone, lets the model sort each finding. Noise falls and real defects stand out.

B · Why this does not meet the full requirement

The bot should attach a confidence score to each comment and the pipeline should suppress anything the model rates below 0.8.

Model confidence on subjective comments is not a reliable filter.

Models tend to report high confidence on their own opinions, so most style comments would pass the threshold. It also adds a second vague judgment on top of the first instead of defining what counts.

C · Why this does not meet the full requirement

The bot needs a hard cap of three comments per pull request, so the most important findings are forced to the top of the list.

A cap truncates output but lets style comments crowd out real defects.

With a cap of three, the bot might pick three spacing comments and drop the missing alt text. A limit on count says nothing about which findings qualify, so it can make the important-defect miss rate worse.

D · Why this does not meet the full requirement

The model is too creative for review work; lowering temperature would make it produce fewer, more consistent comments per pull request.

Lower temperature repeats the same noisy judgments more consistently.

Temperature controls how much wording varies between runs. The model would still decide that palette opinions are worth posting, just more predictably. Noise comes from the criteria, not randomness, so the flood continues.

Check your understanding · explain the design in your own words

Why does "only significant issues" not reduce noise, and what does?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Prompt iteration as engineering Study: Prompting basics

Study: Prompt iteration as engineering Study: Prompting basics Prompting technique choice Temperature Calibration

Return to Q287's situation and clause-by-clause explanation

Q288. Claude is writing a converter from a bank's legacy CSV export into a double-entry ledger JSON. Each revision fixes one problem and breaks another: refunds end up as debits, and the counterparty lands at the top level instead of inside the entry object. The mapping has been described in two paragraphs of prose. What should be supplied next? A) A request that Claude first restate the full mapping in its own words so any misunderstanding surfaces before the next revision. B) A JSON Schema for the ledger entry, with an instruction to validate every converted object against it before returning. C) An expanded specification listing every CSV column, its sign convention and its target path in the ledger object. D) Three real CSV rows, including a refund and a transfer, each paired with the exact ledger JSON object it should produce.

Understand Q288 · the situation and each part of the question

The real situation behind this question

A fintech startup is migrating customers off a legacy bank export into its ledger service. A developer is using Claude to write the converter. The prose spec seemed clear, but each revision trades one bug for another: refunds flip sign, counterparties end up in the wrong place. The developer is tired of the whack-a-mole cycle and wants to give Claude the one artifact that will end it.

Read the question in parts

Part 1 · Each revision fixes one problem and breaks another

The model is reinterpreting prose on each pass, a sign that the specification itself is ambiguous.

Part 2 · described in two paragraphs of prose

Prose has already been tried, so another prose artifact is unlikely to change the outcome.

Part 3 · What should be supplied next?

You are choosing the next input to Claude, not a process change or a different model.

What you are being asked to decide

What best shows the exact mapping when prose keeps being misread? Choose paired examples as the way to specify an ambiguous transformation.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Paired examplesReal inputs shown with their exact expected outputs, making the mapping unambiguous.
JSON SchemaDescribes the allowed shape of the output; it cannot express which input value goes where.
Target shape versus mappingKnowing what the output looks like differs from knowing how each input maps into it.

Study: Where exactly do I put the paired examples? Study: Why asking for understanding is useful but insufficient Few-shot JSON Schema Input versus output schema

Understand Q288's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When the rules have been explained and keep being misapplied, show instead of tell. A few representative CSV rows paired with their exact ledger objects make sign handling and nesting unambiguous and double as regression tests, which neither a schema nor more prose can do.

Connect the design to the situation

A refund row with a negative amount must become a credit to the customer account.

Best answer: D. The rules have been explained and misapplied repeatedly, which means the prose is being interpreted differently than intended. Concrete input-output pairs show the exact sign handling and nesting in one glance, and they double as regression tests for the converter.

What each choice would do here

A · Why this does not meet the full requirement

A request that Claude first restate the full mapping in its own words so any misunderstanding surfaces before the next revision.

A restatement may sound right while the code still maps signs incorrectly.

Claude may restate the rules accurately; it already has them. The failures come from applying the rules to concrete rows, so a correct paraphrase gives false confidence while the code still misplaces fields.

B · Why this does not meet the full requirement

A JSON Schema for the ledger entry, with an instruction to validate every converted object against it before returning.

A schema validates nesting but says nothing about whether a refund is a debit or a credit.

A schema would catch the counterparty at the wrong level, which helps, but a refund recorded as a debit is perfectly schema-valid. The sign convention, the core ambiguity, cannot be expressed as shape, so half the bugs remain.

C · Why this does not meet the full requirement

An expanded specification listing every CSV column, its sign convention and its target path in the ledger object.

More prose adds more text to misinterpret, which is the failure already observed.

A column-by-column table is more precise than paragraphs but still leaves room to misread the sign convention or the nesting. It lengthens the specification without showing a single concrete result.

D · Why this meets the requirement

Three real CSV rows, including a refund and a transfer, each paired with the exact ledger JSON object it should produce.

Correct: concrete pairs pin down sign handling and nesting that prose kept leaving ambiguous.

Claude sees that the row "2024-02-03,-45.00,REFUND,ACME" must become a credit entry with counterparty nested inside the entry object. Refund, transfer and purchase pairs cover the edge cases, and the same pairs become unit tests the converter must pass.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

One input row paired with its exact output

JSON
{
  "input_csv_row": "2024-02-03,-45.00,REFUND,ACME Outdoor",
  "expected_ledger_entry": {
    "date": "2024-02-03",
    "entry": {
      "type": "credit",
      "amount_cents": 4500,
      "counterparty": "ACME Outdoor",
      "memo": "refund"
    }
  }
}

The pair shows the sign convention and the nesting at once. Two more pairs, a purchase and a transfer, complete the picture, and all three become tests for the converter.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What best shows the exact mapping when prose keeps being misread?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Where exactly do I put the paired examples? Study: Why asking for understanding is useful but insufficient

Study: Where exactly do I put the paired examples? Study: Why asking for understanding is useful but insufficient Few-shot JSON Schema Input versus output schema

Return to Q288's situation and clause-by-clause explanation

Q289. A platform team has Claude generate reference entries for internal API endpoints, which a portal renders directly. Across months of use, about a fifth of entries omit the authentication requirement or the list of error responses, even though the system prompt describes both in detail. Every endpoint has an auth mode and at least one error response. Which change guarantees these fields are present? A) Record the documentation standard in the repository instructions file so it applies to every generation session without repeating it in each prompt. B) Add five complete example entries covering different endpoint styles, so the model sees authentication and error responses in every example. C) Define a record_endpoint_doc tool with auth_mode and error_responses as required fields, set strict: true, and force that tool for each entry. D) Run a linter over each generated entry and regenerate any entry that is missing the authentication or error response sections.

Understand Q289 · the situation and each part of the question

The real situation behind this question

An internal developer portal shows reference documentation for hundreds of endpoints, generated by Claude from source code and route definitions. Engineers rely on it to know how to authenticate and what errors to handle. Despite a detailed system prompt, a meaningful share of entries skip one of those two sections, and teams ship clients that ignore error cases. The platform team wants a change that makes omission impossible, not just rarer.

Read the question in parts

Part 1 · which a portal renders directly

The output is machine-consumed structured data, so a schema can govern it without changing the product.

Part 2 · Every endpoint has an auth mode and at least one error response

The source always supports these fields, so requiring them will not force fabrication.

Part 3 · guarantees these fields are present

Only a mechanism that enforces presence qualifies, not one that merely makes it likelier.

What you are being asked to decide

What guarantees that every entry includes auth and error responses? Choose schema enforcement when field presence must be guaranteed and the source always supports it.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Required field in a tool schemaA field the generated arguments must contain, enforced during generation with strict mode.
strict: trueConstrains tool arguments to the schema so a required field cannot be skipped.
Few-shotDemonstrations that raise the likelihood of a pattern but do not enforce it.

Study: An agent can use a new tool, not just new instructions Study: Structured output — the engineering ladder Structured outputs Strict Few-shot

Understand Q289's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When downstream code needs fields every time and the source always has them, make them required in a strict, forced tool schema so omission is impossible. Examples and standing instructions only improve the odds, and linting catches misses after paying for them.

Connect the design to the situation

An endpoint entry for DELETE /orders with no error responses listed.

Best answer: C. Required fields in a strict tool schema make omission structurally impossible, which is the only option here that guarantees presence. Examples and standing instructions improve the odds, and linting detects omissions after the fact at the cost of regenerations.

What each choice would do here

A · Why this does not meet the full requirement

Record the documentation standard in the repository instructions file so it applies to every generation session without repeating it in each prompt.

Moving the instruction to a persistent file changes where it lives, not whether it is enforced.

A persistent instructions file saves repeating the standard, yet it is still an instruction. The same prose that fails a fifth of the time in the system prompt keeps failing when loaded from a file.

B · Why this does not meet the full requirement

Add five complete example entries covering different endpoint styles, so the model sees authentication and error responses in every example.

Examples improve consistency but cannot make omission impossible.

Five good examples will likely lower the omission rate, and they remain useful for teaching tone and detail. But the model can still skip a section on an unusual endpoint, so presence is encouraged rather than guaranteed.

C · Why this meets the requirement

Define a record_endpoint_doc tool with auth_mode and error_responses as required fields, set strict: true, and force that tool for each entry.

Correct: required fields in a strict, forced tool make omission structurally impossible.

With auth_mode and error_responses required, strict mode on, and the tool forced, every call includes both fields. Because every endpoint genuinely has them, the requirement does not push the model to invent values, and examples can still refine quality.

D · Why this does not meet the full requirement

Run a linter over each generated entry and regenerate any entry that is missing the authentication or error response sections.

Linting detects misses afterward and pays for regeneration instead of preventing them.

The linter would catch every omission, but each catch costs a regeneration, and a regenerated entry can omit something else. It is a detective control where a preventive one is available.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Tool that makes the two sections mandatory

JSON
{
  "name": "record_endpoint_doc",
  "description": "Record reference documentation for one API endpoint.",
  "strict": true,
  "input_schema": {
    "type": "object",
    "properties": {
      "path": {"type": "string"},
      "auth_mode": {"type": "string", "enum": ["none", "api_key", "oauth", "mtls"]},
      "error_responses": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {"status": {"type": "integer"}, "meaning": {"type": "string"}},
          "required": ["status", "meaning"],
          "additionalProperties": false
        }
      }
    },
    "required": ["path", "auth_mode", "error_responses"],
    "additionalProperties": false
  }
}

auth_mode even has "none" as a legal value, so an endpoint without auth is expressible rather than forced into a wrong mode.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What guarantees that every entry includes auth and error responses?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: An agent can use a new tool, not just new instructions Study: Structured output — the engineering ladder

Study: An agent can use a new tool, not just new instructions Study: Structured output — the engineering ladder Structured outputs Strict Few-shot

Return to Q289's situation and clause-by-clause explanation

Q290. An SRE team uses Claude to draft incident postmortems as Markdown pages for its wiki. Reviewing past drafts, customer impact is missing from roughly two in five. The prompt says "write a thorough postmortem", and leadership requires customer impact in every postmortem, even minor ones. What is the root cause of the gap? A) The output should become JSON with a required customer_impact field, because only schema enforcement can make a section appear reliably. B) "Thorough" never says which sections are mandatory, so the model decides case by case; name each required section and require it even when the finding is "none". C) The prompt lacks examples; adding two sample postmortems that include customer impact is the only way to establish the expected structure. D) The drafts need a larger model, because deciding what belongs in a thorough postmortem requires stronger judgment than the current one provides.

Understand Q290 · the situation and each part of the question

The real situation behind this question

An SRE team writes a postmortem after every incident, and Claude now drafts them from timelines and chat logs. Leadership reads these pages to understand how outages affect customers. An audit shows that many drafts skip customer impact, especially after smaller incidents, and leadership has made that section non-negotiable. The team wants to know why a request for a thorough postmortem keeps producing incomplete ones.

Read the question in parts

Part 1 · as Markdown pages for its wiki

The deliverable is a human-read document, so the fix should keep it a readable page.

Part 2 · write a thorough postmortem

A vague quality word that leaves the choice of sections to the model on each incident.

Part 3 · even minor ones

The requirement applies to incidents the model is most likely to judge as having no impact worth writing.

What you are being asked to decide

Why do postmortems keep skipping customer impact? Trace coverage gaps to vague qualifiers and fix them with explicit mandatory sections.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Mandatory sectionsA named list of headings that must appear in every draft, even with "none".
Schema enforcementGuarantees fields in structured output; converting a human document to JSON changes the product.
Few-shotExamples that suggest structure; useful, but they imply rather than state what is required.

Study: Prompt iteration as engineering Study: Choose the fix — instructions, examples, schema, or checks? Prompt template Structured outputs Few-shot

Understand Q290's answer · full reasoning and worked examples
Claude synthesized · the point, straight

"Thorough" leaves the section list to the model, which drops customer impact when it judges impact minor. Name every mandatory section in the prompt and require it even when the finding is none, which keeps the wiki page readable and closes the gap.

Connect the design to the situation

A database failover postmortem with timeline and root cause but no impact section.

Best answer: B. The prompt leaves the section list to the model's judgment, and it judges some incidents as having no customer impact worth writing about. Explicitly naming mandatory sections, and requiring them even when empty, fixes the root cause while keeping the Markdown page the wiki needs.

What each choice would do here

A · Why this does not meet the full requirement

The output should become JSON with a required customer_impact field, because only schema enforcement can make a section appear reliably.

Turning a wiki page into JSON changes the deliverable when an explicit prompt already fixes it.

A JSON field would force presence, but the wiki needs a page people read. You would need a renderer and would lose the natural prose flow, when a clearly stated section list solves the problem in the format already in use.

B · Why this meets the requirement

"Thorough" never says which sections are mandatory, so the model decides case by case; name each required section and require it even when the finding is "none".

Correct: the qualifier leaves section choice to the model; naming mandatory sections removes it.

The prompt now lists Summary, Timeline, Customer impact, Root cause and Action items, each required, with "Customer impact: none observed" allowed when true. The model no longer decides whether the section belongs, and the page stays a readable Markdown document.

C · Why this does not meet the full requirement

The prompt lacks examples; adding two sample postmortems that include customer impact is the only way to establish the expected structure.

Examples help, but they only imply the rule; stating it is the direct fix, and not the only way.

Two sample postmortems would likely help, and they pair well with the fix. But they show structure implicitly; the model may still decide a minor incident needs no impact section. Stating the mandatory list is the direct fix.

D · Why this does not meet the full requirement

The drafts need a larger model, because deciding what belongs in a thorough postmortem requires stronger judgment than the current one provides.

The gap comes from an undefined requirement, which a larger model would also have to guess.

A larger model applies its own idea of thoroughness more fluently, but leadership's rule that impact is mandatory is still never stated. It might skip the section less often, yet the requirement remains implicit and costs rise on every draft.

Check your understanding · explain the design in your own words

Why do postmortems keep skipping customer impact?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Prompt iteration as engineering Study: Choose the fix — instructions, examples, schema, or checks?

Study: Prompt iteration as engineering Study: Choose the fix — instructions, examples, schema, or checks? Prompt template Structured outputs Few-shot

Return to Q290's situation and clause-by-clause explanation

Q291. An online retailer's support pipeline classifies each ticket's product_area with a required enum: billing, login, shipping, returns. New kinds of tickets, such as gift card balances and screen-reader problems, are being filed as billing or login, and those queues quietly receive work they cannot handle. Which schema change keeps routing reliable and stops the forced matches? A) Replace the enum with a free-text string so the model can describe any product area in its own words. B) Make product_area nullable so the model returns null for any ticket that does not match one of the four areas. C) Add "other" to the enum plus an other_detail string that the model fills when no listed area fits the ticket. D) Expand the enum to thirty areas covering every ticket type the support team can think of today.

Understand Q291 · the situation and each part of the question

The real situation behind this question

An online retailer routes support tickets to specialized queues based on product area. The original schema listed four areas, which covered almost everything at launch. Since then the retailer has added gift cards and improved accessibility, and those tickets land in billing or login, where agents lack the tools to resolve them. Customers wait longer and agents waste time reassigning. The team wants a schema that absorbs new categories gracefully.

Read the question in parts

Part 1 · a required enum: billing, login, shipping, returns

The schema allows only four values, so any other ticket must be squeezed into one of them.

Part 2 · queues quietly receive work they cannot handle

The damage is silent misrouting, so the fix must make no-fit visible and routable.

Part 3 · keeps routing reliable and stops the forced matches

The right design must preserve enum-based routing and also capture unlisted cases.

What you are being asked to decide

How do you stop unlisted ticket types from being forced into the wrong enum value? Design enums with an escape hatch that keeps both routing and information.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Closed enumA fixed list of allowed values; anything outside it gets forced into the nearest match.
Escape valueAn "other" option that makes no-fit an honest, schema-valid answer.
Null versus otherNull says nothing is known; other plus detail says what it is, even if unlisted.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Closed schema JSON Schema Missing values in schemas

Understand Q291's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A closed list forces every ticket into one of its values, so unlisted types silently misroute. Add an "other" value with a detail string: known areas keep routing deterministically, and new areas become visible, routable and measurable.

Connect the design to the situation

A gift card balance question lands in the billing queue.

Best answer: C. An escape value lets the model admit that no listed area fits, and the detail field records what it actually is, while known areas still route by enum. Free text breaks routing, null loses the information and conflates unknown with unlisted, and a longer list is still closed.

What each choice would do here

A · Why this does not meet the full requirement

Replace the enum with a free-text string so the model can describe any product area in its own words.

Free text lets the model describe tickets but breaks the deterministic routing that enums provide.

The model would write "gift card balance" in one ticket and "gift certificate issue" in another, and routing code would need its own classifier to interpret the text. You regain accuracy by giving up the reliability that motivated the enum.

B · Why this does not meet the full requirement

Make product_area nullable so the model returns null for any ticket that does not match one of the four areas.

Null throws away what the ticket is about and lumps no-fit with genuinely unclear tickets.

Every no-fit ticket becomes null, so a screen-reader problem and a completely unreadable ticket look identical. The team loses the signal about emerging categories and still needs a person to reread each ticket.

C · Why this meets the requirement

Add "other" to the enum plus an other_detail string that the model fills when no listed area fits the ticket.

Correct: the escape value plus detail captures new areas without disturbing known routes.

Known tickets still route by the four values. A gift card ticket becomes other with other_detail "gift card balance inquiry", which routes to a triage queue and gives the team data on which new areas deserve their own enum values.

D · Why this does not meet the full requirement

Expand the enum to thirty areas covering every ticket type the support team can think of today.

A longer list is still closed, so the next new type gets forced again.

Thirty values cover today's ideas, but the next product launch creates a new kind of ticket and forced matching returns. A long list also makes classification harder between near-synonyms, without solving the underlying closed-list problem.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Enum with an escape hatch

JSON
{
  "type": "object",
  "properties": {
    "product_area": {"type": "string", "enum": ["billing", "login", "shipping", "returns", "other"]},
    "other_detail": {
      "type": ["string", "null"],
      "description": "When product_area is other, describe the actual area in a few words; otherwise null."
    }
  },
  "required": ["product_area", "other_detail"],
  "additionalProperties": false
}

Routing reads product_area; analysts read other_detail to decide which new areas earn a permanent enum value.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you stop unlisted ticket types from being forced into the wrong enum value?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Closed schema JSON Schema Missing values in schemas

Return to Q291's situation and clause-by-clause explanation

Q292. A logistics company added error-feedback retries to its bill-of-lading extraction and fixed most failures. Of the records that still fail, some ask for a seal number the document never contains, some repeat the same error of reading "freight collect" as a payment date, and some retries break fields that were already correct. Which plan matches each group? A) Accept null and queue the missing seal numbers for review; add targeted examples for the "freight collect" error; retry only the failing field, keeping validated fields fixed. B) Raise the retry limit to five for all three groups and write more detailed error messages, giving the model more chances to correct every failure. C) Stop retrying altogether and route every remaining failure to manual review, so nothing unresolved reaches the carrier systems. D) Make every field nullable across the schema; send the recurring error to a larger model; raise temperature on retries so they explore different outputs.

Understand Q292 · the situation and each part of the question

The real situation behind this question

A logistics company extracts bills of lading to feed carrier systems. Adding retries with error feedback fixed most failures, but a stubborn remainder still fails. The analyst sorted them and found three distinct groups with different causes. The operations director wants a single plan, but the analyst suspects one uniform change will not fit all three, and the team needs a mapping from cause to remedy.

Read the question in parts

Part 1 · a seal number the document never contains

This group has absent source data, so the only honest outcomes are null and human follow-up.

Part 2 · repeat the same error of reading "freight collect" as a payment date

A consistent misreading that feedback alone has not fixed needs a demonstration of the correct reading.

Part 3 · retries break fields that were already correct

Regressions appear when the retry re-extracts everything, so the retry scope should shrink.

What you are being asked to decide

What should you do with each kind of failure that retries did not fix? Diagnose residual failures by cause and assign a matching remedy to each.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Scoped retryA retry that asks only for the failing field so correct fields are not re-extracted and broken.
Targeted examplesA few demonstrations of the exact misreading, showing the correct interpretation.
Failure diagnosisSorting failures by root cause before choosing a fix.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Instructions, examples, and JSON are different layers Validation and repair Few-shot Failure diagnosis

Understand Q292's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Residual failures after retries have different causes, so they need different fixes. Absent data goes to null and review, a repeated misreading gets targeted examples, and regressions are stopped by retrying only the failing field.

Connect the design to the situation

"Freight collect" recorded as a payment date on every retry.

Best answer: A. Each group has a different cause: absent data cannot be retried into existence, a repeated misreading needs a demonstration of the right reading, and regressions come from re-extracting everything instead of just the failing field. One uniform response fits none of them well.

What each choice would do here

A · Why this meets the requirement

Accept null and queue the missing seal numbers for review; add targeted examples for the "freight collect" error; retry only the failing field, keeping validated fields fixed.

Correct: each group gets the remedy that addresses its specific cause.

Missing seals become null with a review ticket, so nobody invents numbers. Two or three examples show that "freight collect" is a payment term, not a date. Retries now request only the failing field and merge it into the validated record, so good fields stay good.

B · Why this does not meet the full requirement

Raise the retry limit to five for all three groups and write more detailed error messages, giving the model more chances to correct every failure.

More attempts cannot create absent data and repeat the same misreading.

Five retries on a missing seal number either fail five times or eventually fabricate one. The "freight collect" error has already survived detailed feedback, and more full re-extractions give more chances to break good fields.

C · Why this does not meet the full requirement

Stop retrying altogether and route every remaining failure to manual review, so nothing unresolved reaches the carrier systems.

Blanket manual review ignores two groups that have cheap automated fixes.

Manual review is safe but costly, and two of the three groups have reliable automated fixes. Sending everything to people spends reviewer time on errors that examples or scoped retries would resolve.

D · Why this does not meet the full requirement

Make every field nullable across the schema; send the recurring error to a larger model; raise temperature on retries so they explore different outputs.

Blanket nullability weakens good fields, and temperature adds noise rather than fixes.

Blanket nullability lets the model return null for fields that are always present, lowering coverage. A larger model may or may not fix the misreading, and higher temperature makes regressions more likely rather than less.

Check your understanding · explain the design in your own words

What should you do with each kind of failure that retries did not fix?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Choose the fix — instructions, examples, schema, or checks? Study: Instructions, examples, and JSON are different layers

Study: Choose the fix — instructions, examples, schema, or checks? Study: Instructions, examples, and JSON are different layers Validation and repair Few-shot Failure diagnosis

Return to Q292's situation and clause-by-clause explanation

Q293. A food-labeling startup extracts nutrition panels with a strict tool schema, so every result parses and every field has the right type. Auditors still find panels where calories do not match the fat, carbohydrate and protein amounts, and sodium recorded in the potassium field. A developer proposes moving the schema from the tool into output_config.format. Which statement is exactly right? A) Both routes guarantee valid shape and types only; recompute calories from macros, cross-check mineral placement, and retry with the specific error. B) Adding more required fields, such as a calories_check field, would force the model to reconcile the macros and place each mineral correctly. C) Strict mode is being bypassed, because a working strict schema would reject a panel whose calories disagree with its macronutrients. D) Moving the schema to output_config.format would fix the errors, because structured outputs check values against the source while tool schemas only check types.

Understand Q293 · the situation and each part of the question

The real situation behind this question

A food-labeling startup builds a database of nutrition facts from product photos for grocery apps. Engineers were confident after enabling strict schemas: no more parse failures. Then an auditor found panels with impossible calorie counts and minerals in the wrong fields. A developer suggests that the other structured output route would be stricter. The team needs to decide whether that helps or whether the gap is somewhere else.

Read the question in parts

Part 1 · every result parses and every field has the right type

Shape and type guarantees are already fully in place, so the remaining errors lie elsewhere.

Part 2 · calories do not match the fat, carbohydrate and protein amounts

A relationship between values, which only arithmetic validation after extraction can check.

Part 3 · moving the schema from the tool into output_config.format

A switch between two routes that provide the same kind of structural guarantee.

What you are being asked to decide

Does strict or native structured output catch wrong values, and if not, what does? Recognize the limits of schema enforcement and place semantic validation after it.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Structured outputsGeneration constrained to a schema for shape and types, via strict tools or output_config.format.
output_config.formatRequest setting that constrains the JSON answer text to a schema; another route to the same shape guarantee.
Semantic validationChecks on meaning, such as recomputing totals, that run after the structured result arrives.

Study: Output handling — trust the shape, check the content Study: Structured output — the engineering ladder Structured outputs output_config Validation and repair

Understand Q293's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Strict tool arguments and output_config.format both guarantee structure, not truth. Switching routes or adding fields leaves meaning unchecked, so recompute and cross-check values after extraction and retry with the specific error.

Connect the design to the situation

A panel listing 250 calories when the macros imply about 180.

Best answer: A. Strict tool arguments and output_config.format are two routes to the same kind of guarantee: parseable output matching the schema's structure. Neither knows nutrition arithmetic or which mineral is which, so semantic checks after extraction, with targeted retries, are required.

What each choice would do here

A · Why this meets the requirement

Both routes guarantee valid shape and types only; recompute calories from macros, cross-check mineral placement, and retry with the specific error.

Correct: shape is guaranteed either way; meaning needs recomputation and cross-checks.

Code recomputes calories as roughly nine per gram of fat and four per gram of carbohydrate and protein, compares within a tolerance, and checks sodium and potassium placement. Failures go back with the specific finding, and persistent ones are flagged for review.

B · Why this does not meet the full requirement

Adding more required fields, such as a calories_check field, would force the model to reconcile the macros and place each mineral correctly.

More required fields add more shape, not correctness, and a self-reported check can be wrong too.

A calories_check field would be filled by the same model that made the mistake, and it could simply say the numbers agree. Adding required fields increases the shape to fill without adding any independent verification of meaning.

C · Why this does not meet the full requirement

Strict mode is being bypassed, because a working strict schema would reject a panel whose calories disagree with its macronutrients.

Strict mode is working; disagreement between valid numbers is outside what it enforces.

Strict mode is doing exactly its job: the output parses and matches the declared types. A panel where calories disagree with macros is still a valid object under the schema, so there is nothing for strict mode to reject.

D · Why this does not meet the full requirement

Moving the schema to output_config.format would fix the errors, because structured outputs check values against the source while tool schemas only check types.

Both routes constrain structure; neither compares values with the source.

output_config.format constrains the model's JSON answer text to the schema, just as strict constrains tool arguments. Neither compares values with the photo, so the same calorie mismatches and swapped minerals would appear in the new route.

Check your understanding · explain the design in your own words

Does strict or native structured output catch wrong values, and if not, what does?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Output handling — trust the shape, check the content Study: Structured output — the engineering ladder

Study: Output handling — trust the shape, check the content Study: Structured output — the engineering ladder Structured outputs output_config Validation and repair

Return to Q293's situation and clause-by-clause explanation

Q294. A commercial landlord extracts lease terms into a portfolio model. The schema requires annual_escalation_pct as a number. About a quarter of leases say only "rent escalates per the CPI schedule in Addendum C", and the addendum is not part of the file; for those, the model returns figures such as 3.0. Which schema change and instruction fix this? A) Add an escalation_basis text field next to the number so the model can record "CPI per Addendum C" when no percentage is given. B) Keep the field required and add an extraction_confidence number so the portfolio model can down-weight escalations the model is unsure about. C) Instruct the model to enter 0.0 when the escalation is defined elsewhere, and add a needs_addendum boolean set to true for those leases. D) Allow annual_escalation_pct to be null, and instruct the model to return null unless a percentage is explicitly stated in the lease text provided.

Understand Q294 · the situation and each part of the question

The real situation behind this question

A commercial landlord models rent growth across hundreds of leases to plan financing. Lease terms are extracted into a portfolio model, and escalation rates drive the forecasts. Many leases tie escalation to an inflation schedule in an addendum that was never scanned. The extracted records show neat percentages for those leases anyway, and the finance team realizes their forecast rests on invented numbers. They need a change that stops it without hiding the gap.

Read the question in parts

Part 1 · requires annual_escalation_pct as a number

The schema forces a numeric value on every lease, even when the text does not state one.

Part 2 · the addendum is not part of the file

The true value is outside the provided input, so any number returned was not read from the source.

Part 3 · Which schema change and instruction fix this?

The answer must combine a structural change with guidance on when to use it.

What you are being asked to decide

How do you stop the model inventing escalation rates that live in a missing addendum? Pick schema design plus instruction over placeholders and confidence scores for referenced-but-absent values.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Nullable fieldA field that can hold null, giving "not stated in this text" a legal representation.
Sentinel placeholderA fake value such as 0.0 standing for missing; downstream math treats it as real.
Self-reported confidenceA score the model assigns itself; it does not remove the invented value and is often poorly calibrated.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Missing values in schemas JSON Schema Calibration

Understand Q294's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When the value lives in a document you did not provide, the schema must allow null and the instruction must say to use it. Placeholders look like real data, confidence scores leave the invented value in place, and side notes do not remove the pressure on a required number.

Connect the design to the situation

"Escalation per Addendum C" becomes annual_escalation_pct: 3.0.

Best answer: D. The percentage is not in the provided text, so the schema must let the model say so, and the instruction must say when to do it. A zero placeholder is a real-looking value that corrupts forecasts, a confidence score leaves the invented number in place, and a side text field does not remove the pressure on the still-required number.

What each choice would do here

A · Why this does not meet the full requirement

Add an escalation_basis text field next to the number so the model can record "CPI per Addendum C" when no percentage is given.

The number is still required, so the model still has to produce one.

Recording the basis is useful context, and it can complement the real fix. But annual_escalation_pct is still a required number, so the model still has to invent one, and the fabrication continues beside the helpful note.

B · Why this does not meet the full requirement

Keep the field required and add an extraction_confidence number so the portfolio model can down-weight escalations the model is unsure about.

The invented figure stays in the record with a score that may not reflect reality.

The record still carries 3.0, which forecasts will use unless every consumer applies the weighting. The model's own confidence may be high on invented numbers, so the score neither removes nor reliably marks the fabrication.

C · Why this does not meet the full requirement

Instruct the model to enter 0.0 when the escalation is defined elsewhere, and add a needs_addendum boolean set to true for those leases.

Zero is a real escalation rate, so forecasts silently assume flat rent.

A zero looks like a legitimate flat-rent lease, and anyone summing or averaging escalation will include it. The boolean helps only if every consumer remembers to check it, so the sentinel quietly skews forecasts toward no growth.

D · Why this meets the requirement

Allow annual_escalation_pct to be null, and instruct the model to return null unless a percentage is explicitly stated in the lease text provided.

Correct: nullable plus a clear instruction gives absence an honest value.

annual_escalation_pct becomes number or null, with the instruction "return null unless a percentage is stated in this lease text". Leases pointing to Addendum C now yield null, which finance can route to someone who retrieves the addendum.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Nullable field with an explicit rule

JSON
{
  "type": "object",
  "properties": {
    "annual_escalation_pct": {
      "type": ["number", "null"],
      "description": "Percentage stated in this lease text. Return null if the lease refers to another document for the rate."
    },
    "escalation_reference": {
      "type": ["string", "null"],
      "description": "Quote the reference, such as CPI schedule in Addendum C, when the rate is defined elsewhere; otherwise null."
    }
  },
  "required": ["annual_escalation_pct", "escalation_reference"],
  "additionalProperties": false
}

The number can now be honestly empty, and the reference field tells finance exactly which document to fetch.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you stop the model inventing escalation rates that live in a missing addendum?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content

Study: Schema design details that prevent hallucination Study: Output handling — trust the shape, check the content Missing values in schemas JSON Schema Calibration

Return to Q294's situation and clause-by-clause explanation

Q295. A bank's onboarding agent must call verify_identity_document before anything else, then choose among sanctions_screen, address_check and open_account, and it must never reply to the orchestrator in prose between steps. Sometimes it jumps straight to address_check, and sometimes it writes a status paragraph instead of calling a tool. Which tool_choice sequence enforces both requirements? A) On the first turn, force {"type": "tool", "name": "verify_identity_document"}; on later turns, use {"type": "any"} so some tool must be called. B) On the first turn, use {"type": "any"}; on later turns, force each specific tool by name in the order the compliance manual lists. C) Use {"type": "any"} on every turn, since that already requires a tool call and the model will naturally start with verification. D) Keep tool_choice on auto and add a hook that rejects any turn in which verify_identity_document was not the first tool called.

Understand Q295 · the situation and each part of the question

The real situation behind this question

A bank is automating customer onboarding with an agent that must verify identity documents before any other check, for regulatory reasons. The orchestrator expects a tool call at every step so it can log and audit actions. In testing, the agent sometimes skips verification or writes a friendly status update instead of acting. The compliance team wants configuration that enforces both the order and the tool-only behavior, not a stronger prompt.

Read the question in parts

Part 1 · must call verify_identity_document before anything else

The first action must be a specific tool, which only a forced named tool_choice guarantees.

Part 2 · must never reply to the orchestrator in prose between steps

Later turns must be tool calls, which tool_choice "any" enforces while leaving the tool choice open.

Part 3 · Which tool_choice sequence enforces both requirements?

The answer changes the setting across turns rather than using one setting throughout.

What you are being asked to decide

Which tool_choice settings force verification first and tool calls afterward? Combine forced and "any" tool_choice across turns to enforce order and tool-only replies.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Forced named tooltool_choice with type tool and a name; the response must be that exact call.
tool_choice anyThe response must call one of the provided tools, but the model picks which.
Agent loopYour code sends results back and decides when to stop, since "any" never allows a final text reply.

Study: JSON wording, tool selection, and strict mode are different Study: Output config during a multi-turn tool loop tool_choice Forced tool extraction Agent loop

Understand Q295's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Order and tool-only replies are two constraints needing two settings. Force the named verification tool on the first turn, then use "any" so the model must call some tool, and let the orchestrator end the loop because "any" never allows a closing text reply.

Connect the design to the situation

The agent calls address_check before the identity document is verified.

Best answer: A. Forcing the named tool is the only setting that fixes which tool runs first, and "any" is the setting that removes prose while leaving the choice of next tool to the model. Your orchestrator then ends the loop itself, for example after open_account returns.

What each choice would do here

A · Why this meets the requirement

On the first turn, force {"type": "tool", "name": "verify_identity_document"}; on later turns, use {"type": "any"} so some tool must be called.

Correct: forced first call fixes order; "any" afterward removes prose while keeping choice.

Turn one can only be verify_identity_document. Afterward, "any" guarantees a call among the remaining tools, and the model decides whether screening or address checks come next. The orchestrator stops the loop after open_account, since "any" never permits a final text reply.

B · Why this does not meet the full requirement

On the first turn, use {"type": "any"}; on later turns, force each specific tool by name in the order the compliance manual lists.

"Any" first does not guarantee verification, and forcing every later step removes needed judgment.

"Any" on the first turn could pick sanctions_screen before verification, which fails the core requirement. Forcing every later step in a fixed order also removes the model's ability to react to results, such as stopping after a failed screen.

C · Why this does not meet the full requirement

Use {"type": "any"} on every turn, since that already requires a tool call and the model will naturally start with verification.

"Any" prevents prose but lets the model pick address_check first.

"Any" guarantees a tool call on every turn, which fixes the prose problem. But the model is free to choose address_check or sanctions_screen first, so the regulatory ordering requirement is left to chance.

D · Why this does not meet the full requirement

Keep tool_choice on auto and add a hook that rejects any turn in which verify_identity_document was not the first tool called.

Auto still allows prose, and a rejection hook reacts after the fact instead of constraining generation.

Auto leaves prose as a valid response, so status paragraphs keep happening. A rejection hook detects a wrong first call after the model has spent a turn, then needs retry logic; it is a detective control where a preventive setting exists.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

tool_choice across turns

JSON
{
  "turn_1": {"tool_choice": {"type": "tool", "name": "verify_identity_document"}},
  "turn_2_onward": {"tool_choice": {"type": "any"}},
  "stop_condition": "orchestrator ends the loop after open_account returns or a check fails"
}

The first request fixes which tool runs; later requests only require that some tool runs. The stop condition lives in your code, not in the model's reply.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Which tool_choice settings force verification first and tool calls afterward?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: JSON wording, tool selection, and strict mode are different Study: Output config during a multi-turn tool loop

Study: JSON wording, tool selection, and strict mode are different Study: Output config during a multi-turn tool loop tool_choice Forced tool extraction Agent loop

Return to Q295's situation and clause-by-clause explanation

Q296. A shipping marketplace extracts parcel weight from seller listings into weight_kg. Sellers write "1.2 kg", "1,200 g", "2 lb 8 oz" and occasionally "two and a half pounds". A conversion script after extraction handles the metric forms but breaks on the imperial and spelled-out variants, and new variants keep appearing. What is the more robust fix? A) Instruct the model to report weight_kg in kilograms, with one short example for each known format, including a spelled-out imperial weight. B) Add a schema pattern that only accepts plain numbers in weight_kg, so any unconverted value fails validation and triggers a retry. C) Extend the conversion script with explicit patterns for every format seen so far, including the spelled-out imperial ones. D) Make weight_kg nullable so the model returns null for any listing written in a format that the script cannot convert reliably.

Understand Q296 · the situation and each part of the question

The real situation behind this question

A shipping marketplace quotes postage from parcel weight in seller listings. Sellers from different countries write weight however they like. The current pipeline extracts the raw text and a script converts it, which handles metric units but breaks on pounds and ounces and on spelled-out numbers. Every week a new variant appears and another listing gets a wrong quote. The team wants a fix that does not require chasing each new format in code.

Read the question in parts

Part 1 · breaks on the imperial and spelled-out variants

Post-processing only works for patterns someone anticipated, so it fails on natural-language variation.

Part 2 · new variants keep appearing

The solution must generalize to unseen formats, which rules out patching patterns one at a time.

Part 3 · What is the more robust fix?

You must pick the approach that keeps working as formats change, not a one-time patch.

What you are being asked to decide

How do you get consistent kilograms when sellers keep inventing new ways to write weight? Move normalization into the extraction prompt with examples instead of brittle post-processing.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Few-shot normalizationExamples that show each known source format converted to the target unit.
Deterministic converterCode that handles only formats it was written for and fails on new ones.
Pattern constraintA schema rule that rejects values in the wrong form but does not teach the conversion.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Deterministic JSON Schema

Understand Q296's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A converter written for known patterns will always trail new variants. Tell the model the target unit and show one example per known format, so conversion happens at extraction time and generalizes to phrasings no one anticipated.

Connect the design to the situation

"Two and a half pounds" must become weight_kg 1.13.

Best answer: A. The model reads language and can convert units at extraction time if it is told the target and shown the known variants. A deterministic script handles only the patterns someone anticipated, a validation pattern rejects bad output without teaching conversion, and nullability discards weights that are present.

What each choice would do here

A · Why this meets the requirement

Instruct the model to report weight_kg in kilograms, with one short example for each known format, including a spelled-out imperial weight.

Correct: the instruction sets the target and the examples teach each variant at extraction time.

The prompt says "return weight_kg as a number in kilograms" and shows "1,200 g" → 1.2, "2 lb 8 oz" → 1.13 and "two and a half pounds" → 1.13. The model generalizes to new phrasings because it understands units, not just patterns.

B · Why this does not meet the full requirement

Add a schema pattern that only accepts plain numbers in weight_kg, so any unconverted value fails validation and triggers a retry.

A pattern rejects bad values but gives the model no guidance on converting them.

Unconverted values would fail and trigger retries, but the retry has no guidance on converting "2 lb 8 oz". The model may keep returning the same raw text or guess a conversion, so failures persist at added cost.

C · Why this does not meet the full requirement

Extend the conversion script with explicit patterns for every format seen so far, including the spelled-out imperial ones.

Every new variant needs another code change, so the script is always one step behind.

Adding patterns fixes today's known formats, but the next seller writes "about 3 pounds" or "1 kilo" and the script breaks again. The team is locked into an endless cycle of code patches trailing behind reality.

D · Why this does not meet the full requirement

Make weight_kg nullable so the model returns null for any listing written in a format that the script cannot convert reliably.

Null discards weights that are present, breaking shipping quotes for those listings.

The weight is right there in the listing, just in an unfamiliar format. Returning null means those parcels get no quote or a default, which harms sellers and buyers, and the conversion problem is simply moved to a human.

Check your understanding · explain the design in your own words

How do you get consistent kilograms when sellers keep inventing new ways to write weight?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples?

Study: Instructions, examples, and JSON are different layers Study: Where exactly do I put the paired examples? Few-shot Deterministic JSON Schema

Return to Q296's situation and clause-by-clause explanation

Drill · Tools & MCP (Q297–Q317)

Q297. An agent is moving a service's config file off a retired database host. In deploy/values.yaml the string db-old.corp.internal appears 31 times, and every one of them must become db-new.corp.internal. The agent calls Edit with that old string and the new one, and Edit refuses because the old string is not unique in the file. What is the right next call? A) Call Edit 31 times, each with enough neighbouring lines in old_string that only one occurrence matches per call B) Glob for '**/db-old' to find the files involved, then Edit each file it returns C) Repeat the same Edit call with replace_all set to true, since every occurrence in this file is meant to change D) Read the whole file, then Write it back with the new host typed into each of the 31 places

Understand Q297 · the situation and each part of the question

The real situation behind this question

A platform team is decommissioning an old database server. Every service keeps its connection settings in a values file, and one service's file mentions the old hostname 31 times across replicas, jobs and health checks. The agent doing the migration hits Edit's uniqueness refusal on its first attempt. Choosing the wrong recovery either burns dozens of tool calls or risks rewriting a production config by hand.

Read the question in parts

Part 1 · every one of them must become

The change is intended at all occurrences, which is exactly the condition under which replace_all is safe.

Part 2 · Edit refuses because the old string is not unique

The failure is a uniqueness check on a normal call, not a sign that Edit is the wrong tool.

What you are being asked to decide

Edit said the text is not unique, but you want to change all of it. What do you send next? Know when replace_all is the correct Edit input rather than many single edits or a whole-file rewrite.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
replace_allan Edit input that replaces every exact occurrence of old_string in the file instead of requiring one unique match
not uniqueEdit's refusal when old_string matches more than once and replace_all was not set
Globa tool that matches file paths by name pattern, not by the text inside files

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Edit Read Glob

Understand Q297's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A plain Edit must match once, so a repeated string triggers a refusal. When every occurrence should change, replace_all is the purpose-built input and finishes in one call. Many anchored edits or a whole-file rewrite reach the same place with more cost and risk.

Connect the design to the situation

A hostname repeated 31 times in one YAML file must all change together.

Best answer: C. Edit refused because a plain call must match exactly once; replace_all is the input that says every exact match is intended, so the same call succeeds in one step. Thirty-one context-widened edits reach the same result with far more calls and more chances to slip.

What each choice would do here

A · Why this does not meet the full requirement

Call Edit 31 times, each with enough neighbouring lines in old_string that only one occurrence matches per call

It works, but it turns one intended change into 31 calls, each needing a hand-built unique anchor.

Each call would need old_string widened with neighbouring lines until it matches once, and repeated YAML blocks make unique anchors hard to find. Thirty-one calls also multiply the chance that one anchor is wrong and a replica keeps the old host.

B · Why this does not meet the full requirement

Glob for '**/db-old' to find the files involved, then Edit each file it returns

Glob matches file names, the file is already known, and a hostname inside a file never appears in its path.

Glob looks at paths, so a pattern built from the hostname finds nothing unless a file happens to be named after it. The agent already knows the exact file, so a file search adds a step without solving the refusal.

C · Why this meets the requirement

Repeat the same Edit call with replace_all set to true, since every occurrence in this file is meant to change

The refusal only means more than one match exists; replace_all tells Edit that all of them should change.

Edit with replace_all true swaps every exact occurrence of db-old.corp.internal in values.yaml in one operation. Because the requirement is that all 31 change, there is no occurrence to protect, so the uniqueness check that blocked the first call is no longer needed.

D · Why this does not meet the full requirement

Read the whole file, then Write it back with the new host typed into each of the 31 places

Rewriting the whole file to change one repeated string resends everything and risks damaging unrelated lines.

Read then Write regenerates the entire file from the model's copy. It can work, but any accidental change to indentation or an unrelated key ships with it, and it costs far more output than a targeted replacement.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

The retry that succeeds

JSON
{
  "tool": "Edit",
  "input": {
    "file_path": "deploy/values.yaml",
    "old_string": "db-old.corp.internal",
    "new_string": "db-new.corp.internal",
    "replace_all": true
  }
}

Same old and new strings as the refused call; only replace_all changes, declaring that every match is intended.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Edit said the text is not unique, but you want to change all of it. What do you send next?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Edit Read Glob

Return to Q297's situation and clause-by-clause explanation

Q298. A fare-comparison orchestrator sends a subagent to collect prices for 40 flight routes. At route 26 the airline portal starts rejecting requests because the subagent's session token has expired. The orchestrator decides whether to refresh credentials, reassign work, or report a partial comparison. Which TWO things should the subagent's error report contain? (Select TWO.) A) A permission failure category (expired session) and the request that was rejected B) A decision to abandon the whole comparison, since the subagent is the component closest to the failure C) The prices already collected for routes 1–25, plus the list of routes still uncovered D) A success status carrying the 25 routes, so the comparison is not marked as failed

Understand Q298 · the situation and each part of the question

The real situation behind this question

A travel startup compares fares by fanning out to subagents that log into airline partner portals. Mid-run, one subagent's session expires after 25 routes. The orchestrator could refresh the token and resume, hand the remaining 15 routes to another worker, or publish a comparison flagged as incomplete. Which of those it can choose depends entirely on what the failing subagent reports back.

Read the question in parts

Part 1 · The orchestrator decides whether to refresh credentials, reassign work, or report a partial comparison

The decision sits upstream, so the report must carry the facts that decision needs, not the decision itself.

Part 2 · the subagent's session token has expired

This is a permission-type failure; retrying the identical request will fail until credentials change.

What you are being asked to decide

When a helper agent fails halfway, what should it tell the boss agent? Identify the error context a coordinator needs to choose a recovery path.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Failure categorya label such as transient, validation, permission or business that tells the orchestrator which recovery can work
Partial resultsthe 25 route prices already gathered, which the orchestrator can keep instead of refetching
Orchestratorthe coordinating agent that decides whether to refresh, reassign or report partially

Study: The MCP error contract: isError and structured error metadata Subagent Orchestrator Error taxonomy

Understand Q298's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The coordinator makes the recovery choice, so the failing worker reports facts rather than verdicts: the failure category, what it was attempting, and what it already finished. Hiding the failure or deciding locally both take that choice away.

Connect the design to the situation

A price-collecting subagent loses its session after 25 of 40 routes.

Best answer: AC. The orchestrator owns recovery, so it needs to know what went wrong and what was being attempted, and it needs the finished work so nothing is fetched twice. Reporting success hides the gap, and making the abort decision inside the subagent takes the choice away from the component that has the whole picture.

What each choice would do here

A · Why this does not meet the full requirement

A permission failure category (expired session) and the request that was rejected

A permission category plus the rejected request tells the orchestrator that refreshing credentials is the fix.

Categorizing the failure as permission, with the rejected request attached, points directly at the remedy: refresh the session and resume. A bare 'request failed' would leave the orchestrator guessing between rate limits, bad input and outages.

B · Why this does not meet the full requirement

A decision to abandon the whole comparison, since the subagent is the component closest to the failure

Abandoning is a recovery decision that belongs to the orchestrator, which can see options the subagent cannot.

The subagent sees only its own slice. The orchestrator may hold a fresh token or spare workers, so letting the subagent abort the whole comparison discards recoverable work and bypasses the component responsible for that call.

C · Why this does not meet the full requirement

The prices already collected for routes 1–25, plus the list of routes still uncovered

Returning completed work and the remaining gap lets the orchestrator resume from route 26 rather than restart.

With routes 1–25 priced and 26–40 listed as uncovered, the orchestrator can merge what exists and send only the remainder to a retry or another worker. Without it, the whole batch is refetched and the portal is hit twice as hard.

D · Why this does not meet the full requirement

A success status carrying the 25 routes, so the comparison is not marked as failed

Labelling a partial run as success hides the 15 missing routes from the component that must act on them.

A success status tells the orchestrator the job is done, so the published comparison silently omits 15 routes. Customers see an incomplete price list presented as complete, which is worse than an honest failure.

Check your understanding · explain the design in your own words

When a helper agent fails halfway, what should it tell the boss agent?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The MCP error contract: isError and structured error metadata

Study: The MCP error contract: isError and structured error metadata Subagent Orchestrator Error taxonomy

Return to Q298's situation and clause-by-clause explanation

Q299. In a 2,000-line legacy test file, Claude Code needs to change the timeout inside one specific retry block. The same five-line block is pasted in several tests, so Edit returns a match-not-unique error. Only the block inside test_refund_partial should change. Which recovery changes exactly that block without restructuring the file? A) Repeat the Edit with replace_all set to true so the uniqueness refusal no longer triggers at all B) Move each test into its own file so every copy of the block becomes unique per file C) Read the full file, change that one block in memory, and Write it back D) Use Bash with sed to rewrite the timeout on a line number read from an earlier grep

Understand Q299 · the situation and each part of the question

The real situation behind this question

A payments team keeps an old integration test file that grew by copy and paste. The refund tests all share an identical retry block. A flaky partial-refund test needs a longer timeout, but the other refund tests are tuned and must not move. Claude Code's first Edit attempt fails because the block's text appears several times, and the developer needs a recovery that touches only the right test.

Read the question in parts

Part 1 · Only the block inside `test_refund_partial` should change

Every other copy must stay as it is, which rules out any replace-everything approach.

Part 2 · without restructuring the file

The fix must be a content change, not a reorganization of the test suite to suit the tool.

What you are being asked to decide

Edit can't find a unique spot. How do you change just the one copy you mean? Choose a recovery for a non-unique Edit that keeps the change scoped to one occurrence.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Non-unique matchEdit found old_string more than once and refuses rather than guess which copy was intended
Read + Write fallbackload the full file, change the intended location, and write the complete file back
seda shell stream editor run through Bash; it edits by pattern or line number with no uniqueness check

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Edit Write Bash

Understand Q299's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A non-unique match means Edit cannot know which copy you meant, so it refuses. When only one copy should change, the reliable recovery is to Read the full file, apply the change in the right place, and Write it back. replace_all is for when all copies should change.

Connect the design to the situation

The same retry block appears in several tests; only one test's timeout should change.

Best answer: C. Edit refused because it cannot tell which copy was meant. Reading the whole file and writing it back with the change placed in the correct test is the documented fallback that stays targeted. replace_all silences the refusal by changing every copy, which is exactly what must not happen here.

What each choice would do here

A · Why this does not meet the full requirement

Repeat the Edit with replace_all set to true so the uniqueness refusal no longer triggers at all

replace_all removes the uniqueness check by changing every copy, so unrelated tests get the new timeout too.

Setting replace_all makes Edit succeed by changing the timeout in every refund test at once. The flaky test gets its longer timeout, and the tuned tests silently get it too, which can mask real slowness elsewhere.

B · Why this does not meet the full requirement

Move each test into its own file so every copy of the block becomes unique per file

Restructuring the test suite to satisfy a text match is a disproportionate change for a one-line edit.

Splitting tests into many files would make each copy unique per file, but it churns imports, fixtures and review history to solve a one-line problem. The requirement explicitly rules out restructuring.

C · Why this meets the requirement

Read the full file, change that one block in memory, and Write it back

Working from the full file lets the change land in the one named test while every other copy stays untouched.

Reading the full file gives the agent the complete context, so it can find the copy inside test_refund_partial and change only that timeout before writing the file back. It is slower than a clean Edit but deterministic about which copy changes.

D · Why this does not meet the full requirement

Use Bash with sed to rewrite the timeout on a line number read from an earlier grep

Line numbers go stale as soon as anything shifts, and sed gives no feedback if it hits the wrong line.

A line number taken from an earlier search is only right until something above it moves. sed will happily rewrite whatever is on that line, and nothing tells the agent if it edited the wrong block.

Check your understanding · explain the design in your own words

Edit can't find a unique spot. How do you change just the one copy you mean?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Edit Write Bash

Return to Q299's situation and clause-by-clause explanation

Q300. A telehealth intake assistant must call record_consent before any clinical conversation starts. Afterwards it should answer questions freely, calling other tools only when useful. In testing, the first reply is sometimes a friendly greeting with no tool call. Which request setting guarantees the consent call on the opening request without constraining later turns? A) Send only record_consent in the opening request's tools list and leave tool_choice on auto B) Keep tool_choice on auto and add a system prompt line saying consent must always be recorded first C) Use tool_choice type any on every request so some tool is always called before any text is produced D) Opening request: tool_choice naming record_consent; every later request: auto

Understand Q300 · the situation and each part of the question

The real situation behind this question

A telehealth company must log patient consent before its assistant discusses symptoms, and auditors will check that no clinical exchange happens first. The team already tells the model to record consent, yet test transcripts show occasional warm greetings with no tool call. They need a request-level guarantee for the opening turn only, because later turns must feel like a normal conversation.

Read the question in parts

Part 1 · guarantees the consent call on the opening request

A guarantee means an API setting, not an instruction the model usually follows.

Part 2 · without constraining later turns

Whatever forces the first call must be removed afterwards so text replies and free tool choice return.

What you are being asked to decide

How do you force one tool on the first turn only? Tell apart a guarantee (tool_choice) from a hint (prompt text or a trimmed tool list).

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
tool_choicethe request field that controls whether Claude may, must, or must use a particular tool on this call
autoClaude decides whether to call a tool or reply in text
anyClaude must call some tool but may choose which one

Study: Tool use: the Claude API mechanics tool_choice System prompt tool_use

Understand Q300's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Only tool_choice turns a preference into a guarantee. Naming record_consent forces that call on the opening request, and switching to auto afterwards restores normal behaviour. Prompt text and trimmed tool lists remain hints under auto, and any on every turn over-constrains the whole conversation.

Connect the design to the situation

A consent tool must run before anything else, then the assistant behaves normally.

Best answer: D. Naming the tool in tool_choice forces that exact call on that request, which is the only setting here that is a guarantee rather than a strong hint. Returning to auto afterwards restores normal behaviour. Setting any on every request forces some tool call on every turn, which blocks plain answers and does not guarantee the consent tool specifically.

What each choice would do here

A · Why this does not meet the full requirement

Send only record_consent in the opening request's tools list and leave tool_choice on auto

With auto, Claude may still reply in text even when only one tool is offered, so the greeting problem remains.

Offering a single tool narrows what can be called but does not require a call. Under auto the model can still answer with a greeting, so the exact failure from testing can recur.

B · Why this does not meet the full requirement

Keep tool_choice on auto and add a system prompt line saying consent must always be recorded first

A prompt instruction raises the odds but leaves a text-only first reply possible, which is the failure seen in testing.

The system prompt already pushes toward consent, and testing shows the model sometimes greets first anyway. Instructions shape probability; they cannot promise a call, and an auditor needs a guarantee on every opening turn.

C · Why this does not meet the full requirement

Use tool_choice type any on every request so some tool is always called before any text is produced

It forces a tool call every turn, so later plain answers become impossible and consent is still not the guaranteed choice.

type any obliges Claude to call some tool on every request. On turn one it could pick another tool, and on later turns the assistant cannot simply answer a question in text, which breaks the conversational requirement.

D · Why this meets the requirement

Opening request: tool_choice naming record_consent; every later request: auto

A named tool in tool_choice forces that call on that request, and auto afterwards restores free selection.

The opening request sends tool_choice {type: tool, name: record_consent}, so the response is that tool call. Every later request sends {type: auto}, and Claude again chooses between tools and text as the conversation needs.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Opening request versus later requests

JSON
{
  "opening_request": {
    "tool_choice": {
      "type": "tool",
      "name": "record_consent"
    }
  },
  "later_requests": {
    "tool_choice": {
      "type": "auto"
    }
  }
}

The field changes between requests; the tools list can stay the same throughout.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you force one tool on the first turn only?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Tool use: the Claude API mechanics

Study: Tool use: the Claude API mechanics tool_choice System prompt tool_use

Return to Q300's situation and clause-by-clause explanation

Q301. A pharmacy assistant checks refill status with lookup_prescription. For one caller it gets an empty list, reformats the prescription number and tries four more times, then pages a pharmacist. The pharmacist finds the number was never issued. The same empty list is also what the tool returns when the pharmacy database is unreachable. What is the root cause of the wasted retries and the page? A) The model's sampling temperature is too high, which makes it keep trying new number formats B) The escalation policy pages a pharmacist too early and should allow several more lookup attempts first C) The retry limit is set too high; cutting it to two attempts would have avoided most of the wasted work D) Its result gives a valid no-match and a failed lookup the same shape, so they look identical

Understand Q301 · the situation and each part of the question

The real situation behind this question

A retail pharmacy chain added an assistant to its refill line. Callers often read prescription numbers wrong or quote numbers that were never issued. Each time, the lookup tool hands back an empty list, the assistant assumes something broke, tries alternative formats, and eventually pages a busy pharmacist. Staff complain that most pages are for numbers that simply do not exist.

Read the question in parts

Part 1 · The same empty list is also what the tool returns when the pharmacy database is unreachable

The tool's output carries no distinction between no-match and failure, which is the design flaw.

Part 2 · What is the root cause

The question asks for the underlying design problem, not a tuning knob that reduces symptoms.

What you are being asked to decide

Why did the agent keep retrying a lookup whose honest answer was 'no such prescription'? Diagnose an ambiguous tool result contract as the cause of retry loops.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Valid empty resultthe lookup ran successfully and found nothing; the answer is 'no such prescription'
Access failurethe lookup could not run, for example the database was unreachable; retrying may help
Escalationhanding the case to a human pharmacist, which should be reserved for cases the agent cannot resolve

Study: The MCP error contract: isError and structured error metadata tool_result Error taxonomy Escalation

Understand Q301's answer · full reasoning and worked examples
Claude synthesized · the point, straight

An empty list that means both 'nothing matched' and 'could not look' leaves the agent unable to trust a valid answer. Separate the two in the tool contract, success-with-no-results versus a flagged error, and the retries and pages disappear. Retry and escalation tuning only scale the waste.

Connect the design to the situation

An empty list means both 'not found' and 'database down'.

Best answer: D. An empty list means two different things in this tool, so the agent reasonably treats it as a possible failure and retries. The fix belongs in the tool contract: a successful empty result for no match and a flagged error for an unreachable database. Tuning retry counts or escalation timing only changes how much waste occurs.

What each choice would do here

A · Why this does not meet the full requirement

The model's sampling temperature is too high, which makes it keep trying new number formats

Temperature does not create the ambiguity; a perfectly deterministic agent would face the same unclear signal.

Lower temperature might make the reformatting attempts more repetitive, but the agent would still see an empty list that could mean failure. The ambiguity lives in the tool's output, not in the model's sampling.

B · Why this does not meet the full requirement

The escalation policy pages a pharmacist too early and should allow several more lookup attempts first

Waiting longer before paging adds more wasted lookups; the page was only needed because the answer was misread.

Delaying escalation means more reformatting attempts before the pharmacist is paged. The pharmacist then discovers the same thing: the number never existed. It moves the waste around rather than removing it.

C · Why this does not meet the full requirement

The retry limit is set too high; cutting it to two attempts would have avoided most of the wasted work

Fewer retries reduces waste but still treats a correct answer as a failure every time it happens.

Lowering the retry cap to two makes each misread case cheaper, but the agent still retries a correct answer and still escalates it. The cause, an ambiguous result, is untouched, so the pattern continues at smaller scale.

D · Why this meets the requirement

Its result gives a valid no-match and a failed lookup the same shape, so they look identical

Two meanings share one shape, so the agent has no signal that the empty list was already the true answer.

Once the tool returns a success result for 'no prescription with this number' and an error-flagged result for 'database unreachable', the agent can tell the caller the number is not on file after one call and reserve retries for real outages.

Check your understanding · explain the design in your own words

Why did the agent keep retrying a lookup whose honest answer was 'no such prescription'?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The MCP error contract: isError and structured error metadata

Study: The MCP error contract: isError and structured error metadata tool_result Error taxonomy Escalation

Return to Q301's situation and clause-by-clause explanation

Q302. A localization agent pushes 400 UI strings through an MCP translation server. Around string 120 the upstream vendor starts answering HTTP 429 with a Retry-After of 45 seconds, and the server passes back only 'Translation failed'. The agent concludes the service is broken and stops. Which server-side change lets the agent finish the batch on its own? A) Hold each call open inside the server, retrying quietly until the vendor accepts, however long that takes B) Mark the failure as a permission error that is not retryable, so the agent stops sending requests to the vendor C) Return isError true, category transient, retryable true, and a wait hint from Retry-After D) Return the source string untranslated for any string that hits the limit, so the batch always completes

Understand Q302 · the situation and each part of the question

The real situation behind this question

A game studio uses an agent to translate interface text before each release. The translation vendor enforces a per-minute quota. When the quota trips, the in-house MCP server flattens the vendor's detailed 429 response into a generic failure string. The agent, seeing repeated failures, reasonably gives up with 280 strings untranslated, and a localization lead has to restart the job by hand every release.

Read the question in parts

Part 1 · answering HTTP 429 with a Retry-After of 45 seconds

The upstream already knows the failure is temporary and how long it lasts; the server is discarding that information.

Part 2 · lets the agent finish the batch on its own

The fix must give the agent enough structure to pause and resume without a human.

What you are being asked to decide

The vendor says 'slow down for 45 seconds'. How should the server tell the agent? Map a rate-limit response to structured, actionable error metadata.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
429the HTTP status for too many requests; it is temporary, not a broken service
isRetryablean application-defined flag telling the agent that the same call can succeed later
Retry-Afterthe vendor's hint for how long to wait, which the server can forward as a wait field

Study: The MCP error contract: isError and structured error metadata Rate limiting MCP server Backpressure

Understand Q302's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A rate limit is temporary and comes with a known wait. Passing that through as a transient, retryable error with a wait hint lets the agent pause and resume. A generic string, a wrong category, a hidden wait or silent substitution each leave the agent unable to recover correctly.

Connect the design to the situation

A translation vendor returns 429 with Retry-After partway through a batch.

Best answer: C. A 429 is a temporary condition with a known wait, and the agent can only act on that if the server says so in structured form. Transient plus retryable plus a wait hint lets the agent pause and continue. Holding calls open hides the condition and risks client timeouts with no feedback, and substituting untranslated text silently corrupts the output.

What each choice would do here

A · Why this does not meet the full requirement

Hold each call open inside the server, retrying quietly until the vendor accepts, however long that takes

Hiding the limit behind open-ended waits can exceed client timeouts and gives the agent no way to plan.

Blocking inside the server can hold a single call for minutes. The MCP client or agent loop may time out, and the agent cannot tell a slow vendor from a hung server, so it may still abandon the batch.

B · Why this does not meet the full requirement

Mark the failure as a permission error that is not retryable, so the agent stops sending requests to the vendor

A permission label is the wrong category; the agent would stop permanently on a limit that clears in 45 seconds.

Labelling a quota hit as a non-retryable permission failure tells the agent the request can never succeed. It would stop exactly as it does now, only with a more confident and more misleading reason.

C · Why this meets the requirement

Return isError true, category transient, retryable true, and a wait hint from Retry-After

It names the condition, says a retry will work, and says when, which is everything the agent needs to resume.

The server returns an error result with isError true plus fields such as errorCategory transient, isRetryable true and retryAfterMs 45000. The agent waits, resumes at string 120, and completes the batch. These field names are an application convention layered on MCP's isError flag.

D · Why this does not meet the full requirement

Return the source string untranslated for any string that hits the limit, so the batch always completes

Silently shipping English strings as translations turns a visible delay into a hidden quality defect.

Falling back to the source string makes the batch look complete, but players in other languages see English text. The failure moves from a visible error to a shipped defect that someone must find later.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

The error result the agent can act on

JSON
{
  "isError": true,
  "content": [
    {
      "type": "text",
      "text": "{\"errorCategory\": \"transient\", \"isRetryable\": true, \"retryAfterMs\": 45000, \"message\": \"Vendor rate limit reached at string 120 of 400\"}"
    }
  ]
}

isError is the MCP flag; errorCategory, isRetryable and retryAfterMs are application-defined metadata carried in the result content.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

The vendor says 'slow down for 45 seconds'. How should the server tell the agent?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The MCP error contract: isError and structured error metadata

Study: The MCP error contract: isError and structured error metadata Rate limiting MCP server Backpressure

Return to Q302's situation and clause-by-clause explanation

Q303. A hotel chain's book_room MCP tool fails in four ways: (1) the payment gateway times out during a traffic spike, (2) the agent sends a check-out date earlier than the check-in date, (3) the stay exceeds the chain's 28-night limit, (4) the server's credential has no write access for a newly acquired property. Which classification lets the agent react correctly to each? A) (1) permission, not retryable; (2) validation, retryable; (3) business, retryable; (4) permission, not retryable B) (1) transient, retryable; (2) validation, not retryable; (3) validation, not retryable; (4) business, not retryable C) (1) transient, retryable; (2) business, not retryable; (3) validation, not retryable; (4) transient, retryable D) (1) transient, retry; (2) validation, no retry; (3) business, no retry; (4) permission, no retry

Understand Q303 · the situation and each part of the question

The real situation behind this question

A hotel group lets a concierge agent book rooms directly. Its booking server surfaces four recurring failures, and today they all come back as the same generic error, so the agent retries bad dates, apologizes for timeouts as if the guest did something wrong, and never tells staff that a new property is missing write access. The architect is defining the error metadata so each case gets the right response.

Read the question in parts

Part 1 · the stay exceeds the chain's 28-night limit

The request is well formed; a business rule refuses it, so the agent should explain rather than retry.

Part 2 · the server's credential has no write access

This is an access problem that only an administrator can fix, so it needs escalation, not waiting.

What you are being asked to decide

Four different failures: which label and retry flag fits each? Assign error categories by cause so each failure drives a different agent response.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Validationthe input itself is wrong, so the agent must correct the request before trying again
Businessthe request is well formed but a rule refuses it; retrying cannot change the rule
Permissionthe caller lacks access; a human or an admin action must grant it

Study: The MCP error contract: isError and structured error metadata Error taxonomy MCP Escalation

Understand Q303's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Categories exist so each failure produces a different move: wait and retry, fix the input, explain the rule, or escalate for access. Label by cause, and mark only transient failures retryable unchanged.

Connect the design to the situation

A booking tool faces a timeout, bad dates, a policy cap and missing access.

Best answer: D. A timeout clears on its own, so it is transient and worth retrying. Reversed dates are malformed input the agent must fix, a stay-length cap is a policy decision, and a missing write grant needs someone to change access. Mapping the dates as business and the policy as validation swaps the agent's two different responses: fix the input versus explain the policy.

What each choice would do here

A · Why this does not meet the full requirement

(1) permission, not retryable; (2) validation, retryable; (3) business, retryable; (4) permission, not retryable

A gateway timeout is not an access problem, and marking policy refusals retryable invites pointless loops.

Calling a gateway timeout a permission error sends a transient outage to administrators. Marking the date error and the policy cap retryable encourages the agent to resend the same failing request, wasting calls without changing the result.

B · Why this does not meet the full requirement

(1) transient, retryable; (2) validation, not retryable; (3) validation, not retryable; (4) business, not retryable

The 28-night cap is a policy rule, not bad input, and missing access is a permission problem, not a business rule.

Treating the night cap as validation suggests the agent can adjust a field to fix it, when the guest actually needs a different offer. Calling missing access a business rule hides that an admin action would resolve it immediately.

C · Why this does not meet the full requirement

(1) transient, retryable; (2) business, not retryable; (3) validation, not retryable; (4) transient, retryable

It swaps the date error and the policy cap, and treats missing access as something that clears by waiting.

Labelling reversed dates as business tells the agent the request was refused by policy, so it may apologize instead of fixing the dates. Marking missing access transient sends the agent into retry loops that can never succeed until someone grants the permission.

D · Why this meets the requirement

(1) transient, retry; (2) validation, no retry; (3) business, no retry; (4) permission, no retry

Each failure is labelled by its cause, and only the timeout is marked as worth retrying unchanged.

The agent retries the timeout after a pause, corrects the dates and resubmits, tells the guest about the 28-night rule and offers alternatives, and escalates the access gap to staff. Each category maps to a distinct and correct next move.

Check your understanding · explain the design in your own words

Four different failures: which label and retry flag fits each?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The MCP error contract: isError and structured error metadata

Study: The MCP error contract: isError and structured error metadata Error taxonomy MCP Escalation

Return to Q303's situation and clause-by-clause explanation

Q304. A team is retiring the feature flag legacy_checkout_v1 in a large monorepo and does not know which files read it. An agent proposes: (1) Glob '/checkout' to list candidate files, (2) Read each result, (3) Edit the references it finds. Reviewers point out the plan will miss references. What should replace the first step? A) Glob '/*.{ts,py}' instead, so every source file is listed and then read one by one B) Grep for the literal string legacy_checkout_v1 across the whole repository C) Run a single sed command through Bash that rewrites the flag everywhere, skipping the search entirely D) Read the flag registry file first and follow its import statements out to every consumer by hand

Understand Q304 · the situation and each part of the question

The real situation behind this question

An e-commerce company is removing a years-old checkout flag. It is read in the web app, in background jobs, in configuration files and in tests, often by files whose names never mention checkout. An agent's first plan starts with a name-based file search, and reviewers suspect that half the references would be left behind, which would leave dead code paths still switching on a flag nobody sets.

Read the question in parts

Part 1 · does not know which files read it

The unknown is location by content, which is the question Grep answers and Glob cannot.

Part 2 · Reviewers point out the plan will miss references

The flaw is in the discovery step; the later Read and Edit steps are sound once the right files are known.

What you are being asked to decide

You know the text but not the files. Which tool finds the files? Choose content search over name matching when the unknown is where a string is used.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Grepsearches inside files for text or a pattern and returns the files and lines that match
Globmatches file paths against a name pattern; it never looks inside files
Literal stringthe exact flag name, which is the most reliable search key for its references

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Grep Glob Edit

Understand Q304's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When you know the text but not where it lives, search contents: Grep for the literal flag, then Read and Edit each hit. Glob only sees names, so a name-based first step misses every reference in an unrelated-looking file.

Connect the design to the situation

A feature flag is read from files whose names have nothing to do with checkout.

Best answer: B. The unknown is which files contain the flag, and Grep answers exactly that by searching file contents. Glob matches paths, so a flag read by a file named payment_router.ts is invisible to a checkout-named pattern. Listing every source file with Glob is technically complete but forces the agent to read thousands of files to do Grep's job.

What each choice would do here

A · Why this does not meet the full requirement

Glob '**/*.{ts,py}' instead, so every source file is listed and then read one by one

It lists every source file but then relies on reading them all, which is the slow manual version of a content search.

Globbing all TypeScript and Python files guarantees no file is skipped, but it returns thousands of paths with no indication of which contain the flag. The agent then reads them one by one, spending huge context to do a content search manually.

B · Why this meets the requirement

Grep for the literal string legacy_checkout_v1 across the whole repository

A content search on the exact flag name returns every file and line that mentions it, whatever the file is called.

Grep for legacy_checkout_v1 returns exactly the files and lines that mention the flag, whether the file is called payment_router.ts or jobs.yaml. The agent then Reads context around each match and Edits it, so every change is reviewable.

C · Why this does not meet the full requirement

Run a single sed command through Bash that rewrites the flag everywhere, skipping the search entirely

A blind repository-wide rewrite gives no list to review and can alter generated, vendored or unrelated files.

A single sed across the monorepo might rewrite the flag, but it also touches lock files, vendored code and snapshots, and it produces no list for reviewers. Retiring a flag usually means deleting branches, not renaming a string.

D · Why this does not meet the full requirement

Read the flag registry file first and follow its import statements out to every consumer by hand

Tracing imports by hand misses code that reads the flag by name from configuration rather than importing it.

Starting from the registry finds consumers that import a constant, but many services read flags by string name from config or an API. Those references never appear in an import chain, so the manual trace is incomplete and slow.

Check your understanding · explain the design in your own words

You know the text but not the files. Which tool finds the files?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Grep Glob Edit

Return to Q304's situation and clause-by-clause explanation

Q305. An observability team's search_logs tool is described only as 'Searches application logs by regex.' The backing engine uses the RE2 syntax, which has no lookahead or backreferences. The agent picks the tool at the right moments, but many calls fail because it writes patterns such as (?=timeout). Where does the fix belong? A) In the error handling: return the engine's raw parser message so the agent can learn from each failure B) In a server layer that rewrites any lookahead pattern into an equivalent RE2 pattern before it runs C) In each agent's system prompt, as a paragraph explaining exactly which regex features the log engine accepts D) In the tool and pattern-parameter descriptions: state RE2, what it lacks, and a working example

Understand Q305 · the situation and each part of the question

The real situation behind this question

An observability team exposes its log search to several internal agents. The engine was chosen for predictable performance, and its regex flavour rejects lookaheads and backreferences. Agents, trained on many regex dialects, routinely write patterns that the engine refuses, so incident investigations stall on syntax errors. Selection is fine; the model simply has never been told which dialect this tool speaks.

Read the question in parts

Part 1 · The agent picks the tool at the right moments

Selection is working, so the problem is how arguments are written, not which tool is chosen.

Part 2 · described only as 'Searches application logs by regex.'

The definition never names the dialect, so the model falls back on the syntax it sees most often.

What you are being asked to decide

The tool is chosen correctly but fed the wrong regex flavour. Where do you say which flavour it takes? Place input-format requirements in the tool definition, not in other layers.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Tool descriptionthe text in the tool definition that the model reads when deciding how to call the tool
Parameter descriptionthe description inside input_schema for the pattern field, the closest place to where the value is written
Dialectthe specific syntax the backing engine accepts, here RE2 regular expressions

Study: Writing tool definitions that score points Tool description input_schema Tool selection

Understand Q305's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The model writes tool arguments from the tool's definition. When selection is right but syntax is wrong, the missing fact is the input dialect, so it goes in the description and parameter text. Translation layers, prompt copies and error-driven learning all sit further from the decision.

Connect the design to the situation

A log-search tool accepts RE2 patterns but receives lookaheads.

Best answer: D. The model chooses and fills a tool's arguments from its definition, so the input dialect belongs in the description and parameter text it reads at that moment. Rewriting lookaheads into RE2 is not possible in general, because RE2 deliberately lacks those features. Teaching through failures costs a failed call every time.

What each choice would do here

A · Why this does not meet the full requirement

In the error handling: return the engine's raw parser message so the agent can learn from each failure

Clearer errors help debugging, but each lesson still costs a failed call and nothing stops the next one.

Passing through the parser message helps the agent correct a call after it fails, which is worth doing. But it teaches only after a wasted call, every session, and does not prevent the first mistake.

B · Why this does not meet the full requirement

In a server layer that rewrites any lookahead pattern into an equivalent RE2 pattern before it runs

RE2 omits lookaround by design, so many such patterns have no equivalent to translate into.

A rewrite layer would need to turn arbitrary lookarounds into RE2, which is often impossible because the feature is missing on purpose. It adds a complex component that fails on exactly the patterns it exists to handle.

C · Why this does not meet the full requirement

In each agent's system prompt, as a paragraph explaining exactly which regex features the log engine accepts

A prompt paragraph must be copied into every agent and drifts from the tool it describes.

A system prompt paragraph can work for one agent, but every new agent that uses the tool must copy it, and it silently drifts when the engine changes. The knowledge lives away from the tool it describes.

D · Why this meets the requirement

In the tool and pattern-parameter descriptions: state RE2, what it lacks, and a working example

The requirement sits exactly where the model reads it while writing the pattern, for every agent that uses the tool.

Stating 'RE2 syntax; no lookahead, lookbehind or backreferences; example: timeout.*upstream' in the description and pattern field puts the constraint where the model writes the argument. Every agent that connects gets it automatically, and failures drop without any extra infrastructure.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Tool definition that names the dialect

JSON
{
  "name": "search_logs",
  "description": "Searches application logs with an RE2 regular expression. RE2 does not support lookahead, lookbehind or backreferences. Example pattern: timeout.*upstream",
  "input_schema": {
    "type": "object",
    "properties": {
      "pattern": {
        "type": "string",
        "description": "RE2 pattern; no (?=...), (?<=...) or \\1"
      }
    },
    "required": [
      "pattern"
    ]
  }
}

The constraint appears in both the tool description and the parameter description the model reads while writing the argument.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

The tool is chosen correctly but fed the wrong regex flavour. Where do you say which flavour it takes?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Writing tool definitions that score points

Study: Writing tool definitions that score points Tool description input_schema Tool selection

Return to Q305's situation and clause-by-clause explanation

Q306. A support engineering team wants Claude Code to read and update tickets in the widely used helpdesk platform they subscribe to. One engineer has already sketched a custom MCP server covering twelve endpoints. The tech lead pauses the work. Which reason for pausing is best grounded? A) Calling the helpdesk REST API through Bash would avoid MCP entirely and be simpler to maintain B) MCP servers cannot write to external systems, so updating tickets would need a separate integration regardless C) A custom server should be registered in each engineer's user scope first, and that has not been planned D) A maintained server for that platform may already exist; evaluate it before building one

Understand Q306 · the situation and each part of the question

The real situation behind this question

A support engineering group lives in a popular hosted helpdesk. They want Claude Code to triage and update tickets while they work. An enthusiastic engineer has started a custom MCP server with a dozen endpoints, which the team would then have to maintain as the helpdesk API evolves. The tech lead wants to know whether that build is even necessary before more time goes into it.

Read the question in parts

Part 1 · the widely used helpdesk platform

Popular platforms are the most likely to already have maintained MCP servers.

Part 2 · Which reason for pausing is best grounded

The question asks which objection reflects sound integration practice, not which one merely sounds cautious.

What you are being asked to decide

Before writing your own integration, what should you check? Apply build-versus-adopt judgment to MCP integrations.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Maintained serveran MCP server for a service published and kept current by the vendor or community
Build versus adoptthe decision to write your own integration only when existing ones fall short
Team-specific gapsworkflows unique to this team that a general server does not cover

Study: Configuring MCP in Claude Code MCP server Build versus buy .mcp.json

Understand Q306's answer · full reasoning and worked examples
Claude synthesized · the point, straight

For a widely used service, the first step is to check for a maintained MCP server and test it against the team's workflows. Build custom tools only for the gaps. Scope, Bash shortcuts and false limits on MCP writes answer different questions.

Connect the design to the situation

A team starts coding a server for a popular helpdesk that may already have one.

Best answer: D. Popular services often already have maintained MCP servers, and adopting one avoids building and owning code for operations that are already solved. Custom work is then limited to team-specific gaps. Routing through raw API calls in Bash throws away tool definitions and discovery rather than saving effort.

What each choice would do here

A · Why this does not meet the full requirement

Calling the helpdesk REST API through Bash would avoid MCP entirely and be simpler to maintain

Raw API calls lose typed tool definitions and discovery, and push request formatting onto the model.

Shelling out to curl works for one-off calls, but the model must remember endpoints, headers and payload shapes, and nothing is discoverable as tools. It trades a maintained integration for fragile prompt-driven requests.

B · Why this does not meet the full requirement

MCP servers cannot write to external systems, so updating tickets would need a separate integration regardless

MCP tools can perform actions, including writes, so this objection is simply false.

MCP tools are model-controlled operations and routinely create, update and delete records in external systems. Pausing for this reason would rest on a misunderstanding, and the team might build a second integration for no reason.

C · Why this does not meet the full requirement

A custom server should be registered in each engineer's user scope first, and that has not been planned

Scope decides who receives a configuration, not whether the server should be built at all.

Scope choices (local, project or user) decide who gets a server configuration. They matter after the team picks a server, but they say nothing about whether custom code is needed. A shared team tool would usually be project scope anyway.

D · Why this meets the requirement

A maintained server for that platform may already exist; evaluate it before building one

Adopting a maintained server for standard operations avoids owning code that already exists.

The team evaluates the existing server against its real workflows. If it handles reading and updating tickets, they configure it and move on. If one workflow is missing, they add a small custom tool for that gap rather than twelve endpoints.

Check your understanding · explain the design in your own words

Before writing your own integration, what should you check?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Configuring MCP in Claude Code

Study: Configuring MCP in Claude Code MCP server Build versus buy .mcp.json

Return to Q306's situation and clause-by-clause explanation

Q307. A reply-drafting subagent in an airline's customer service system hands control back to the orchestrator whenever it needs a flight's current status, adding about two round trips per ticket. Logs show 80% of these requests are plain status lookups; the rest involve rebooking or compensation decisions. Which design removes most of the overhead while keeping authority where it belongs? A) Stop checking flight status while drafting and let a human verify the draft before it is sent B) A read-only get_flight_status tool on the subagent; rebooking and compensation stay central C) Give the drafting subagent the full reservations toolset so it never needs to hand control back to the orchestrator D) Cache flight statuses in the orchestrator so each round trip returns faster

Understand Q307 · the situation and each part of the question

The real situation behind this question

An airline's service desk uses a hub-and-spoke agent system. The drafting subagent writes replies to passengers, and nearly every reply needs the flight's current status. Today it must ask the orchestrator, which calls the operations tool and passes the answer back. Replies take noticeably longer than agents expected, and the team is tempted to hand the drafter more power to speed things up.

Read the question in parts

Part 1 · 80% of these requests are plain status lookups

The frequent case is a simple read, which is safe to delegate with a narrow tool.

Part 2 · keeping authority where it belongs

Consequential actions like rebooking and compensation must remain with the orchestrator.

What you are being asked to decide

The writer keeps asking the boss to look things up. How do you cut that without over-empowering it? Scope a new capability to the frequent simple case and keep consequential actions central.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Scoped toola tool that performs only the narrow operation needed, here reading one flight's status
Round tripcontrol passing from subagent to orchestrator and back, adding latency each time
Authoritywhich agent may take consequential actions such as rebooking or paying compensation

Study: Reuse implementation without exposing unrestricted tools Least privilege Subagent Hub-and-spoke

Understand Q307's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Round trips for simple reads are best removed by giving the specialist a narrow, read-only tool for exactly those reads. Consequential actions stay with the coordinator. Caching keeps the hops, and broad toolsets trade latency for over-privilege.

Connect the design to the situation

A drafting agent needs flight status often but must not rebook passengers.

Best answer: B. The frequent case is a simple read, so a narrow read-only tool removes those round trips while the rare consequential actions stay with the orchestrator. Handing over the whole reservations toolset also removes round trips but gives a drafting agent authority to rebook and compensate, which breaks least privilege.

What each choice would do here

A · Why this does not meet the full requirement

Stop checking flight status while drafting and let a human verify the draft before it is sent

Dropping the check and adding human review swaps a fast automated lookup for a slower manual one.

Removing the status check risks replies with wrong information, and a human verification step adds more delay than the round trips it replaces. It trades a solvable automation problem for manual labour.

B · Why this meets the requirement

A read-only get_flight_status tool on the subagent; rebooking and compensation stay central

The common read happens in place, and actions with money or bookings attached stay under central control.

The drafter calls get_flight_status directly for the 80% of simple cases and finishes the reply without leaving its turn. When a ticket needs rebooking or compensation, it still hands off, so money and seat changes are decided centrally.

C · Why this does not meet the full requirement

Give the drafting subagent the full reservations toolset so it never needs to hand control back to the orchestrator

It removes round trips by granting rebooking and compensation powers to an agent that only writes replies.

With the full toolset, the drafter could rebook passengers or grant compensation while writing a reply. Round trips disappear, but so does the separation that keeps consequential actions reviewable and controlled in one place.

D · Why this does not meet the full requirement

Cache flight statuses in the orchestrator so each round trip returns faster

A cache speeds each hop but keeps every hop, so the drafting agent still waits on the orchestrator.

Caching speeds the orchestrator's answer, but the drafter still stops, hands off, and waits for control to return. The round trips, which are the overhead, remain; only one part of each trip gets shorter.

Check your understanding · explain the design in your own words

The writer keeps asking the boss to look things up. How do you cut that without over-empowering it?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Reuse implementation without exposing unrestricted tools

Study: Reuse implementation without exposing unrestricted tools Least privilege Subagent Hub-and-spoke

Return to Q307's situation and clause-by-clause explanation

Q308. Nine analysts want the same internal metrics MCP server, reached over HTTP, available in every clone of their reporting repository. Each analyst has a personal API key that the server checks on every request, and security forbids any key in git. Which configuration meets both requirements? A) Commit .mcp.json with one shared service key, relying on the repository being private B) Commit .mcp.json with a header of Bearer ${METRICS_API_KEY}; each analyst sets that variable C) Commit .mcp.json with a placeholder key string that each analyst replaces in their own clone and never commits D) Each analyst runs claude mcp add in local scope with their own key pasted into the command

Understand Q308 · the situation and each part of the question

The real situation behind this question

A finance analytics group builds its monthly reports in a shared repository. The metrics service they query issues each analyst a personal key so every request can be traced to a person. The group wants Claude Code to see the metrics server automatically in every clone, and the security team has already rejected one pull request that contained a key. The architect has to make sharing and secrecy work together.

Read the question in parts

Part 1 · available in every clone of their reporting repository

The definition must be committed and shared, which points to project scope in .mcp.json.

Part 2 · security forbids any key in git

The committed file may name a variable but must never contain the secret itself.

Part 3 · a personal API key

Each analyst supplies their own value, so the config must read it from the individual's environment.

What you are being asked to decide

How does a team share one MCP server config while each person keeps their own key out of git? Combine project scope with environment-variable expansion for shared, secret-free MCP configuration.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Project scopethe server is defined in .mcp.json at the repository root and shared with everyone who clones it
${VAR} expansionClaude Code substitutes an environment variable's value into the config when it loads, so the file holds only the name
Local scopea server entry private to one person in one project, stored outside the repository

Study: Configuring MCP in Claude Code Study: MCP resources and config scoping .mcp.json MCP connection Settings precedence

Understand Q308's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Sharing and secrecy are separate layers: project-scoped .mcp.json shares the server definition through git, and ${VAR} expansion supplies each person's secret from their own environment. Per-person local entries lose the shared definition; committed or hand-edited keys lose the secrecy.

Connect the design to the situation

Nine analysts need one metrics server, each with a personal key.

Best answer: B. Project scope in .mcp.json is how a team shares one server definition through git, and ${VAR} expansion fills the header from each analyst's own environment at load time. Local-scope entries keep keys out of git but leave nine hand-built configurations that drift. A placeholder that people edit in place puts real keys one careless commit away from the repository.

What each choice would do here

A · Why this does not meet the full requirement

Commit .mcp.json with one shared service key, relying on the repository being private

A private repository still stores the secret in history, and one shared key removes per-person accountability.

Committing a key to a private repository still stores it in history and in every clone and fork. A single shared key also means the metrics service cannot tell which analyst made which request, defeating the reason for personal keys.

B · Why this meets the requirement

Commit .mcp.json with a header of Bearer ${METRICS_API_KEY}; each analyst sets that variable

One committed definition for everyone, with each analyst's key read from their own environment at load time.

The committed .mcp.json names the server, its URL, and a header whose value is Bearer ${METRICS_API_KEY}. Each analyst exports their own key in their shell; Claude Code expands the variable when it loads the config. The shared file contains no secret, and the README lists the variable.

C · Why this does not meet the full requirement

Commit .mcp.json with a placeholder key string that each analyst replaces in their own clone and never commits

Editing a tracked file to hold a real key makes an accidental commit of that key likely.

A placeholder that each person overwrites turns a tracked file into a secret store. Git shows the file as modified in every clone, and sooner or later someone stages it with their real key during an unrelated commit.

D · Why this does not meet the full requirement

Each analyst runs claude mcp add in local scope with their own key pasted into the command

Keys stay out of git, but nine separate hand-made entries give the team no single shared definition.

Local-scope entries keep each key in the analyst's own settings, so git stays clean. But the server URL, headers and name live in nine separate places, new analysts must rebuild them by hand, and any change must be repeated nine times.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Committed .mcp.json

JSON
{
  "mcpServers": {
    "metrics": {
      "type": "http",
      "url": "https://metrics.internal.example/mcp",
      "headers": {
        "Authorization": "Bearer ${METRICS_API_KEY}"
      }
    }
  }
}

Only the variable name is committed. Claude Code expands ${METRICS_API_KEY} from each analyst's environment when it loads project servers; ${VAR:-default} supplies a fallback where one makes sense.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How does a team share one MCP server config while each person keeps their own key out of git?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Configuring MCP in Claude Code Study: MCP resources and config scoping

Study: Configuring MCP in Claude Code Study: MCP resources and config scoping .mcp.json MCP connection Settings precedence

Return to Q308's situation and clause-by-clause explanation

Q309. Before changing the signature of charge_card() in a Python billing package, a developer greps the whole repository for charge_card and gets 14 call sites. billing/__init__.py contains from .core import charge_card as submit_charge, and legacy/adapters.py assigns pay = charge_card. What must happen before the call-site list can be trusted? A) Glob for '**/billing.py' to catch files in the billing area that the text search skipped B) Nothing; a repository-wide search on the function name already returns every place it is called C) Search separately for submit_charge and for pay, and include those call sites in the list D) Re-run the same search with case-insensitive matching turned on to catch variant spellings

Understand Q309 · the situation and each part of the question

The real situation behind this question

A payments engineer is about to add a required currency parameter to a core billing function. Changing the signature will break any caller not updated in the same change. Her first search looks complete, fourteen call sites, but the package re-exports the function under a friendlier name and an old adapter module binds it to a short variable. Callers using those names would break in production.

Read the question in parts

Part 1 · gets 14 call sites

This count reflects only lines containing the literal original name, not every caller.

Part 2 · from .core import charge_card as submit_charge

An import-as creates a new name; callers using it never contain the original text.

Part 3 · before the call-site list can be trusted

The question is about completeness of the search, not about the refactor itself.

What you are being asked to decide

The function also goes by other names. What does a single-name search miss? Recognize that literal search misses aliases and plan follow-up searches.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Literal searchGrep finds the exact characters given; it has no idea two names refer to the same function
Aliasanother name bound to the same function, here submit_charge and pay
Call sitea line that actually invokes the function, under whichever name the caller uses

Study: Text search versus symbol references Grep Glob Read

Understand Q309's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Grep finds exact text, so a function reachable under other names needs one search per name. Use the alias lines from the first search to list every name, then search each. File-name patterns and case flags do not bridge different identifiers.

Connect the design to the situation

A function is imported under a new name and assigned to another variable.

Best answer: C. Grep returns lines containing the exact text searched, and a caller writing submit_charge(order) never contains charge_card. The alias lines found in the first search reveal the other names, and each needs its own search. Case-insensitive matching and name-based file lists do not turn one identifier into a different one.

What each choice would do here

A · Why this does not meet the full requirement

Glob for '**/billing.py' to catch files in the billing area that the text search skipped

File names do not reveal which files call the function under another name.

Globbing billing-named files lists paths, not callers. A caller of submit_charge could live in an orders or checkout module whose name never mentions billing, so this step neither finds nor rules out any alias usage.

B · Why this does not meet the full requirement

Nothing; a repository-wide search on the function name already returns every place it is called

The search found lines containing charge_card, not calls made through submit_charge or pay.

The first search did its job: it found every line with the text charge_card, including the two alias definitions. But callers of submit_charge(order) or pay(order) do not contain that text, so they are absent from the fourteen results.

C · Why this meets the requirement

Search separately for submit_charge and for pay, and include those call sites in the list

Each alias is a separate identifier that callers use, so each one needs its own search.

The developer greps for submit_charge and for pay, reads the hits to confirm they are calls to this function, and adds them to the list. Only then does the list reflect every caller that the signature change will break.

D · Why this does not meet the full requirement

Re-run the same search with case-insensitive matching turned on to catch variant spellings

Case changes do not turn submit_charge or pay into the searched name.

Case-insensitive matching would catch Charge_Card or CHARGE_CARD, which are not the problem. submit_charge and pay are entirely different identifiers, so no case setting will match them.

Check your understanding · explain the design in your own words

The function also goes by other names. What does a single-name search miss?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Text search versus symbol references

Study: Text search versus symbol references Grep Glob Read

Return to Q309's situation and clause-by-clause explanation

Q310. A security-review agent has MCP tools scan_secrets, audit_dependencies and check_licenses, each described in about five words. All connect and work. Across 40 reviews the agent used Grep regexes to hunt for credentials and Bash to list packages in most cases, missing secrets that were only in git history. Which change most directly addresses why the MCP tools are passed over? A) Rewrite each MCP description: what it checks, what it returns, when it beats text search B) Reduce the agent to three tools so the choice between them is simpler to make C) Remove Grep and Bash from the agent so the MCP tools are the only way to inspect the repository D) Add a system prompt rule telling the agent to always prefer MCP tools over built-in tools whenever both could apply

Understand Q310 · the situation and each part of the question

The real situation behind this question

A platform security team gave its code-review agent three purpose-built scanners. One inspects git history and entropy, something no regex search can match. Weeks later, review logs show the scanners barely used and a leaked key sitting in an old commit that the agent never found. Everything is connected correctly; the scanners are just described in a handful of words next to richly documented built-ins.

Read the question in parts

Part 1 · each described in about five words

Thin descriptions give the model nothing to weigh against familiar, well-documented alternatives.

Part 2 · All connect and work

Connectivity and configuration are ruled out, pointing at selection signals.

Part 3 · missing secrets that were only in git history

The specialized tool has a real capability advantage that the description never mentions.

What you are being asked to decide

Why does the agent keep using Grep instead of the better scanner, and what fixes it? Diagnose weak descriptions as the cause of under-used specialized tools.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Tool descriptionthe text the model reads to decide whether this tool fits the task better than others
Built-in toolsClaude Code's own tools such as Grep and Bash, which come with thorough documentation
Selection signalwhatever in the tool list tells the model which tool suits the current step

Study: Remove irrelevant tools or explain useful ones Study: What the descriptions should say Tool description Tool selection MCP tools in Claude Code

Understand Q310's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Tools compete on their descriptions. A specialized MCP tool with a five-word description loses to a well-documented built-in even when it is better. Explain what it does, what it returns and when to prefer it; restrict the menu only when a tool is genuinely out of scope.

Connect the design to the situation

A secret scanner described in five words loses to regex search every time.

Best answer: A. The model chooses tools from their descriptions, and five words give no reason to prefer scan_secrets over a familiar Grep. Saying that it inspects git history and high-entropy strings, returns findings with file and commit, and should be used instead of regex search gives the model the reason it lacks. Removing Grep and Bash also cripples legitimate work the review needs.

What each choice would do here

A · Why this meets the requirement

Rewrite each MCP description: what it checks, what it returns, when it beats text search

It gives the model the missing reason to choose each scanner, and when, directly in the definition it reads.

Descriptions like 'Scans current files and full git history for credentials using pattern and entropy checks; returns file, commit and line; use instead of text search when looking for secrets' give the model a concrete reason to choose the scanner at the right moment.

B · Why this does not meet the full requirement

Reduce the agent to three tools so the choice between them is simpler to make

The number of tools is not the problem; the scanners lose because nothing says why they are better.

Cutting to three tools shrinks the menu but says nothing about which of the remaining tools suits which step. If Grep survives the cut, it keeps winning; if it does not, the agent loses a useful tool for no diagnostic reason.

C · Why this does not meet the full requirement

Remove Grep and Bash from the agent so the MCP tools are the only way to inspect the repository

Forcing the choice removes tools the review legitimately needs instead of explaining why the scanners are better.

With Grep and Bash gone, the agent would call the scanners, but it would also lose the ability to read surrounding code or run project commands the review needs. Over-restricting the menu hides a description problem behind a capability loss.

D · Why this does not meet the full requirement

Add a system prompt rule telling the agent to always prefer MCP tools over built-in tools whenever both could apply

A blanket preference rule ignores when each tool fits and does not describe what the scanners actually do.

A preference rule might tilt some choices, but it applies even when Grep is the right tool and still leaves the model unaware that scan_secrets reads git history. It is a blunt override rather than information.

Check your understanding · explain the design in your own words

Why does the agent keep using Grep instead of the better scanner, and what fixes it?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Remove irrelevant tools or explain useful ones Study: What the descriptions should say

Study: Remove irrelevant tools or explain useful ones Study: What the descriptions should say Tool description Tool selection MCP tools in Claude Code

Return to Q310's situation and clause-by-clause explanation

Q311. An agent must raise max_attempts from 3 to 5 in only the payments job of a generated scheduler config. Four job blocks share identical surrounding lines, differing only in a name field twelve lines away. Two Edit attempts with progressively longer old_string both returned a non-unique error. What should the agent do next? A) Read the full file, change max_attempts in the payments block in memory, and Write the file back B) Widen old_string again, this time including twenty surrounding lines on each side of the setting C) Set replace_all to true, then manually change the other three blocks back to 3 with further edits D) Stop and report that Edit cannot handle this file, asking a developer to make the change

Understand Q311 · the situation and each part of the question

The real situation behind this question

An operations team generates its job-scheduler config from a template, so every job block looks the same apart from its name. Payments jobs have started failing on transient network errors and need more retry attempts. The agent asked to make the change keeps getting non-unique errors from Edit because the text around the setting is identical in every block, and it is not clear how far to keep widening.

Read the question in parts

Part 1 · Two Edit attempts with progressively longer old_string both returned a non-unique error

The anchor-widening route has been tried and is not converging, which is the signal to switch methods.

Part 2 · only the `payments` job

Exactly one block should change; the other three must keep their current values.

What you are being asked to decide

Edit keeps failing on identical blocks. What is the reliable fallback? Recognize when to stop anchor-widening and use the Read plus Write fallback.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Anchorthe old_string text Edit uses to locate the change; it must match exactly once
Generated configa file produced by a template, so blocks repeat with near-identical text
Writereplaces a file's full content with new content the agent supplies

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Read Write Edit

Understand Q311's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When repeated near-identical text defeats Edit's unique anchor, stop widening and fall back: Read the whole file, change the intended block, Write it back. replace_all changes every copy, and giving up ignores a reliable path.

Connect the design to the situation

Four identical job blocks in a generated config; only one should change.

Best answer: A. Repeated near-identical blocks make a unique anchor impractical, and two failed widenings confirm it. Reading the whole file lets the agent locate the payments block by its name and change only that value before writing the file back. Replacing everything and reverting three blocks turns one change into four and risks leaving a wrong value behind.

What each choice would do here

A · Why this meets the requirement

Read the full file, change max_attempts in the payments block in memory, and Write the file back

With the full file in view, the agent changes the one block identified by its name and writes the result.

The agent Reads the file, finds the block whose name is payments, changes max_attempts from 3 to 5 there, and Writes the full content back. It works however many identical blocks exist, and the diff shows one changed line.

B · Why this does not meet the full requirement

Widen old_string again, this time including twenty surrounding lines on each side of the setting

Two widenings already failed on identical blocks; a larger anchor becomes fragile and may still not be unique.

Twenty lines each side might finally include the job name, but it produces a large, brittle old_string that must match byte-for-byte. If whitespace or a neighbouring value differs, it fails again, and the agent is still guessing at how much context is enough.

C · Why this does not meet the full requirement

Set replace_all to true, then manually change the other three blocks back to 3 with further edits

Changing all four and reverting three multiplies edits and invites a missed revert.

replace_all would set all four jobs to 5, and the agent would then need three more non-unique edits to put them back, which hit the same anchor problem. One missed revert leaves another job silently retrying more than intended.

D · Why this does not meet the full requirement

Stop and report that Edit cannot handle this file, asking a developer to make the change

A reliable fallback exists, so handing a one-line change to a person is unnecessary.

Reporting failure is honest but premature. Read plus Write is a standard, reliable fallback for exactly this situation, so escalating a one-line config change to a developer wastes human time.

Check your understanding · explain the design in your own words

Edit keeps failing on identical blocks. What is the reliable fallback?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob

Study: Built-in tools: Read, Write, Edit, Bash, Grep, Glob Read Write Edit

Return to Q311's situation and clause-by-clause explanation

Q312. A helpdesk agent has summarize_ticket (description: 'Summarizes tickets') and summarize_thread (description: 'Summarizes threads'). Requests to condense a customer's whole email conversation go to either tool about half the time, and so do requests to summarize one ticket's fields. Which statement correctly identifies the cause? A) Sampling temperature is set too high, so the model flips between two tools it could otherwise reliably tell apart B) The tool list fills too much of the context window, so the model cannot attend to both definitions C) The descriptions give the model no basis for telling the two tools apart D) The API routes the call to one of the two tools at random because their input schemas have the same shape

Understand Q312 · the situation and each part of the question

The real situation behind this question

A support platform added two summarization tools months apart, written by different engineers in a hurry. One condenses a ticket's structured fields; the other condenses a long email conversation. Quality reviewers now see thread summaries that ignore most of the conversation and ticket summaries that invent back-and-forth. Routing logs show roughly even splits for both kinds of request, and nobody can say which tool a new request will hit.

Read the question in parts

Part 1 · 'Summarizes tickets'

A description with no purpose, input format or limits gives the model almost nothing to match against.

Part 2 · Which statement correctly identifies the cause

The question asks for the root cause, not for a workaround that masks it.

What you are being asked to decide

Two tools with almost the same description get picked at random. Why? Diagnose overlapping descriptions as the cause of misrouting.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Tool selectionthe model's choice of which tool to call, driven mainly by the tool definitions it reads
Boundarya statement in a description of what the tool is not for and which tool to use instead
Temperaturea sampling setting that affects variability, not what information the model has

Study: Writing tool definitions that score points Tool description Tool selection Temperature

Understand Q312's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The model chooses tools from their definitions, so two near-identical descriptions yield near-random routing. Give each a distinct purpose, input, output and explicit boundary. Temperature, context size and imagined API routing are not the cause.

Connect the design to the situation

summarize_ticket and summarize_thread differ by one word.

Best answer: C. Claude selects a tool from its name, description and schema, and these two descriptions differ by one word with no stated purpose or boundary. With no distinguishing signal, selection approaches random. The API does not choose tools for the model, so identical schemas cannot cause random routing on their own.

What each choice would do here

A · Why this does not meet the full requirement

Sampling temperature is set too high, so the model flips between two tools it could otherwise reliably tell apart

Lower temperature would make the choice more consistent, not more correct, because the signal is still missing.

At lower temperature the model might pick the same tool more consistently for similar requests, but it could consistently pick the wrong one. Variability is a symptom; the absence of distinguishing information is the cause.

B · Why this does not meet the full requirement

The tool list fills too much of the context window, so the model cannot attend to both definitions

Two short definitions are a trivial share of context; attention is not the constraint here.

Context-window pressure can degrade performance with very large tool sets, but two one-line definitions are negligible. Blaming capacity here would lead to trimming tools without fixing why these two are confused.

C · Why this meets the requirement

The descriptions give the model no basis for telling the two tools apart

One-word differences with no purpose or boundary leave the model nothing to decide on.

Rewriting each description with its purpose, input (a ticket ID versus a thread ID), output, and a boundary such as 'for a full email conversation, use summarize_thread' gives the model clear grounds to route each request to the right tool.

D · Why this does not meet the full requirement

The API routes the call to one of the two tools at random because their input schemas have the same shape

The API does not pick tools; the model does, and schema shape alone does not randomize its choice.

In the Messages API, the model emits a tool_use block naming the tool it chose; there is no server-side router guessing between tools. Two tools can share a schema shape and still be chosen reliably when their descriptions differ clearly.

Check your understanding · explain the design in your own words

Two tools with almost the same description get picked at random. Why?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Writing tool definitions that score points

Study: Writing tool definitions that score points Tool description Tool selection Temperature

Return to Q312's situation and clause-by-clause explanation

Q313. An investment-research system's writing subagent drafts client notes and must confirm details such as closing prices, filing dates and executive names. Each check now goes back through the coordinator to the research subagent, adding several round trips per note. About nine in ten checks are single-value lookups; the rest need multi-source investigation. Which design keeps notes reliable while cutting the overhead? A) Copy the research subagent's complete search and browsing toolset onto the writing subagent so it can check anything itself B) The research subagent pre-fetching every figure it thinks the writer might later need to confirm C) A lookup_fact tool on the writer for single-value checks; multi-source ones still go via the coordinator D) All checks collected during drafting and sent to the coordinator as one batch after the draft is finished

Understand Q313 · the situation and each part of the question

The real situation behind this question

An asset manager uses a multi-agent system to draft client research notes. The writing agent constantly needs to confirm small facts: yesterday's close, when a filing landed, who the current CFO is. Each confirmation detours through the coordinator and the research agent, so a note that should take minutes takes much longer. Compliance insists every figure in a client note is verified before it goes out.

Read the question in parts

Part 1 · About nine in ten checks are single-value lookups

The high-frequency case is narrow and simple, which is what a scoped tool can safely cover.

Part 2 · the rest need multi-source investigation

Complex checks still belong to the research specialist, so the design must keep a path to it.

Part 3 · keeps notes reliable

Reducing overhead must not weaken verification or blur who is responsible for sourcing.

What you are being asked to decide

The writer needs quick fact checks. How do you give them without making it a researcher? Split frequent simple checks from rare complex ones and scope tools accordingly.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Scoped lookupa tool that returns one verified value, such as a closing price, and nothing broader
Capability bloatgiving an agent far more tools than its role needs, which blurs responsibility and raises risk
Role boundarythe division of work where research investigates and writing composes

Study: Reuse implementation without exposing unrestricted tools Study: Where should the logic live Least-privilege tooling Subagent Capability bloat

Understand Q313's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Most checks are simple single-value lookups, so a narrow lookup tool on the writer removes most round trips. Complex investigations still route to the research specialist. Copying the full research toolset, batching checks late, or pre-fetching guesses each trade one problem for another.

Connect the design to the situation

A note writer confirms prices and dates many times per draft.

Best answer: C. A narrow lookup tool handles the frequent single-value checks during drafting, and the rare deep questions still go to the agent built for investigation. Copying the full research toolset also removes round trips, but it turns the writer into a second researcher and blurs responsibility for sourcing quality.

What each choice would do here

A · Why this does not meet the full requirement

Copy the research subagent's complete search and browsing toolset onto the writing subagent so it can check anything itself

It cuts round trips by duplicating the researcher's role inside the writer, bloating its capabilities.

With full search and browsing, the writer could investigate anything itself. Round trips drop, but the writer now does research without the research agent's prompts and checks, and sourcing quality depends on which agent happened to look.

B · Why this does not meet the full requirement

The research subagent pre-fetching every figure it thinks the writer might later need to confirm

Guessing what the writer will need wastes work and still misses whatever the guess did not anticipate.

Pre-fetching likely figures helps when predictions are right, but a note's needs are hard to predict. The research agent does extra work on every note, and unanticipated checks still take the slow path.

C · Why this meets the requirement

A lookup_fact tool on the writer for single-value checks; multi-source ones still go via the coordinator

Frequent simple checks happen in place, while the investigation-heavy minority still reaches the research specialist.

The writer calls lookup_fact for 'AAPL close on the 14th' and continues drafting with a verified value. When a claim needs cross-checking several sources, it still hands off to the coordinator, which sends it to research. Most round trips vanish and roles stay clear.

D · Why this does not meet the full requirement

All checks collected during drafting and sent to the coordinator as one batch after the draft is finished

Batching defers every check until after drafting, so wrong facts shape the note before they are caught.

Batching turns many round trips into one, but only after the draft is written around unverified facts. If a figure is wrong, whole paragraphs built on it must be rewritten, and the final pass still adds latency.

Check your understanding · explain the design in your own words

The writer needs quick fact checks. How do you give them without making it a researcher?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Reuse implementation without exposing unrestricted tools Study: Where should the logic live

Study: Reuse implementation without exposing unrestricted tools Study: Where should the logic live Least-privilege tooling Subagent Capability bloat

Return to Q313's situation and clause-by-clause explanation

Q314. A patent-search tool queries a regional patent office API. Some queries legitimately match nothing; others hit the office's 25-second gateway timeout on complex boolean queries. Both currently come back to the agent as an empty list, and the drafting agent then tells attorneys 'no prior art found' after timeouts. Which pair of responses should the tool return? A) Both as isError false with an empty list, plus a message string saying 'no matches' or 'timed out' for the agent to read B) An empty list for both cases, with the timeouts written to a monitoring log for engineers to investigate later C) No match: isError false, empty list. Timeout: isError true, transient, retryable D) Both as isError true, since an empty result on a patent search is itself something the attorney must review

Understand Q314 · the situation and each part of the question

The real situation behind this question

A patent law firm uses an agent to run prior-art searches before filing. The regional patent office's API often times out on long boolean queries. Because the search tool returns an empty list either way, attorneys have received 'no prior art found' memos for searches that never actually completed. A missed reference can invalidate a patent later, so the firm needs the tool itself to make the difference unmistakable.

Read the question in parts

Part 1 · Both currently come back to the agent as an empty list

The result shape is identical for two different outcomes, which is the defect to fix.

Part 2 · hit the office's 25-second gateway timeout

A timeout is a transient failure that a retry, perhaps with a simpler query, might resolve.

What you are being asked to decide

How should a search tool report 'found nothing' versus 'timed out'? Design result shapes that separate valid empty answers from retriable failures.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
isErrorthe MCP result flag that marks a tool execution as failed; the host maps it to is_error for Claude
Transienta failure expected to clear, such as a timeout, which makes a retry reasonable
Valid empty resultthe search ran correctly and there were genuinely no matching patents

Study: The MCP error contract: isError and structured error metadata tool_result Error taxonomy Graceful degradation

Understand Q314's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A search that ran and found nothing is a success; a search that timed out is a failure. Return them with different error flags, and attach category and retryability to the failure, so the agent reports absence only when absence is real.

Connect the design to the situation

A patent search either matches nothing or hits a gateway timeout.

Best answer: C. A search that ran and found nothing is a successful answer, while a timeout is a failure that may succeed if retried, and only the error flag with structured metadata lets the agent act differently on each. A shared success flag with different prose still forces the agent to interpret free text. Flagging both as errors makes a genuine 'nothing found' look like a failure.

What each choice would do here

A · Why this does not meet the full requirement

Both as isError false with an empty list, plus a message string saying 'no matches' or 'timed out' for the agent to read

The success flag says both worked, leaving the distinction buried in prose the agent must interpret.

The agent sees success with no results in both cases and must parse a message string to tell them apart. Some prompts will read it correctly and some will not, and any downstream code that checks only the flag treats the timeout as a clean search.

B · Why this does not meet the full requirement

An empty list for both cases, with the timeouts written to a monitoring log for engineers to investigate later

The agent still sees identical empty results and keeps reporting no prior art after timeouts.

Logging timeouts helps engineers find a flaky upstream, but the agent's view is unchanged: an empty list after a timeout still looks like no prior art, and memos keep going out wrong until someone reads the logs.

C · Why this meets the requirement

No match: isError false, empty list. Timeout: isError true, transient, retryable

Success for no match and a flagged, categorized, retryable error for the timeout give the agent two distinct signals.

For no match, the tool returns isError false and an empty result list, so the agent can report no prior art with confidence. For a timeout, it returns isError true with errorCategory transient, isRetryable true and a description, so the agent retries or simplifies the query instead of reporting absence.

D · Why this does not meet the full requirement

Both as isError true, since an empty result on a patent search is itself something the attorney must review

Treating a real no-match as an error hides the difference between a clean search and a failed one.

Marking a genuine no-match as an error means every clean search looks like a failure. The agent may retry pointlessly or escalate, and attorneys lose the ability to trust a real 'no prior art' result.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Two results, two meanings

JSON
{
  "no_match": {
    "isError": false,
    "content": [
      {
        "type": "text",
        "text": "{\"results\": [], \"query_completed\": true}"
      }
    ]
  },
  "timeout": {
    "isError": true,
    "content": [
      {
        "type": "text",
        "text": "{\"errorCategory\": \"transient\", \"isRetryable\": true, \"message\": \"Patent office gateway timed out after 25s\"}"
      }
    ]
  }
}

isError is part of the MCP tool result; errorCategory and isRetryable are application-defined fields carried in the content.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How should a search tool report 'found nothing' versus 'timed out'?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The MCP error contract: isError and structured error metadata

Study: The MCP error contract: isError and structured error metadata tool_result Error taxonomy Graceful degradation

Return to Q314's situation and clause-by-clause explanation

Q315. A business-intelligence agent starts nearly every question by calling list_tables, list_dashboards and list_metrics before it can plan a query. These catalogs change about once a week. The team wants the agent to know what data exists from the start, without spending tool calls to find out. Which MCP primitive fits the catalogs? A) The existing listing tools, with their results cached so repeat calls return faster B) MCP prompts, so the catalogs become reusable templates the user can invoke C) A single list_everything tool that returns all three catalogs in one call D) MCP resources, read by the host and supplied as context before planning begins

Understand Q315 · the situation and each part of the question

The real situation behind this question

A retailer's analysts ask a BI agent questions like 'how did returns trend last quarter?'. Before answering anything, the agent lists tables, dashboards and metrics, three calls that return the same catalogs almost every time. Sessions are slow, and the listings crowd the context before any real query runs. The data team wants the agent to begin each question already knowing the landscape.

Read the question in parts

Part 1 · These catalogs change about once a week

Slow-changing reference content suits a resource that the host reads and supplies, rather than a live action.

Part 2 · without spending tool calls to find out

Any design that still requires a discovery call, even a cheaper one, misses the goal.

What you are being asked to decide

The agent keeps listing catalogs before every question. Which MCP feature should carry them? Match catalog-style context to the resource primitive rather than to tools or prompts.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Resourceread-only content a server exposes and the host chooses to read and attach as context
Prompta user-controlled template the server offers, such as a slash command
Exploratory calla tool call made only to discover what exists before real work begins

Study: The three server primitives Study: MCP resources and config scoping MCP resource MCP prompt MCP tool discovery

Understand Q315's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Catalogs describe what data exists and change slowly, which makes them resources: content the host reads and supplies before the agent plans. Tools, cached or merged, still require a discovery call, and prompts are user-invoked templates.

Connect the design to the situation

Table, dashboard and metric catalogs that change weekly.

Best answer: D. Resources are the MCP primitive for application-controlled content: the host reads them and supplies context, so the agent can plan without calling anything first. A merged listing tool or a cache still requires the model to request a tool before it knows what exists. Prompts are user-invoked templates, not a way to show available data.

What each choice would do here

A · Why this does not meet the full requirement

The existing listing tools, with their results cached so repeat calls return faster

Caching makes the calls cheaper but the agent must still decide to make them before it can plan.

Caching makes each listing call return quickly, but latency is only part of the cost. The agent must still choose to call the tools before it knows what exists, so every question still begins with exploration.

B · Why this does not meet the full requirement

MCP prompts, so the catalogs become reusable templates the user can invoke

Prompts are templates a user invokes, which does not put catalog contents in front of the agent automatically.

Exposing catalogs as prompts would give users slash-command templates, perhaps with catalog text baked in. The agent still has no catalog context unless a user invokes the prompt, so autonomous questions keep starting with listing calls.

C · Why this does not meet the full requirement

A single list_everything tool that returns all three catalogs in one call

Merging three calls into one still spends a tool call on discovery every time.

One combined tool reduces three calls to one, which helps a little. But the agent still starts every question with a discovery call, and the large combined result lands in context on every session.

D · Why this meets the requirement

MCP resources, read by the host and supplied as context before planning begins

Resources carry application-controlled content the host can supply up front, so planning starts informed.

The server exposes the three catalogs as resources. The host reads them (or a concise summary index) and attaches that context at the start of the session, so the agent plans its first query immediately. Resources are not auto-injected; the host decides what to read and include.

Check your understanding · explain the design in your own words

The agent keeps listing catalogs before every question. Which MCP feature should carry them?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The three server primitives Study: MCP resources and config scoping

Study: The three server primitives Study: MCP resources and config scoping MCP resource MCP prompt MCP tool discovery

Return to Q315's situation and clause-by-clause explanation

Q316. A contract-review subagent was given a general run_sql tool so it could fetch standard clauses from the clause library table. In production it has also queried the HR salary table and once attempted an UPDATE on a clause. What most effectively stops this? A) Send every run_sql statement to the orchestrator for approval before it executes B) Tell the subagent in its prompt to query only the clause library and never modify any data C) Replace run_sql with get_clause, which takes a clause ID and runs one fixed read-only query D) Keep run_sql but reject any statement that contains UPDATE, DELETE, or the name of the salary table

Understand Q316 · the situation and each part of the question

The real situation behind this question

A legal operations team built a subagent to compare incoming contracts against the firm's standard clauses. For convenience it received the same general SQL tool the analytics agents use. An audit found it reading salary data while answering an unrelated question and, once, trying to update a clause's text. Nothing was damaged, but the firm now needs a control that cannot be talked around.

Read the question in parts

Part 1 · so it could fetch standard clauses

The legitimate need is one narrow read, which defines the tool that should exist.

Part 2 · once attempted an UPDATE on a clause

The current tool permits writes; any fix that leaves arbitrary SQL possible leaves this open.

What you are being asked to decide

The agent's SQL tool lets it do far more than read clauses. How do you stop that for real? Prefer interface-level constraints over prompts and denylists to enforce least privilege.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Purpose-specific toola tool whose inputs and backend allow only the intended operation, here reading one clause
Denylista list of forbidden patterns; anything not listed is allowed, so gaps are bypasses
Least privilegeeach agent receives only the access its role needs and nothing more

Study: Reuse implementation without exposing unrestricted tools Study: MCP security: architect-grade answers Least privilege Allowlist Confused deputy

Understand Q316's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Capability is safest when constrained at the interface. Replace the general SQL tool with a clause-by-ID reader so misuse cannot be expressed. Prompts request, denylists enumerate, and approval gates add latency, while the general tool stays in place.

Connect the design to the situation

A clause-reading agent has a general SQL tool and wanders into HR data.

Best answer: C. A purpose-built tool accepts only a clause ID and runs a fixed read query, so salary lookups and updates are simply not expressible. A keyword denylist tries to enumerate bad SQL and is easy to bypass, and a prompt rule is a request, not a limit.

What each choice would do here

A · Why this does not meet the full requirement

Send every run_sql statement to the orchestrator for approval before it executes

Approval adds a round trip to every legitimate lookup while the tool stays fully general.

Orchestrator approval adds latency to every clause lookup and relies on another model to judge SQL safety. The subagent still holds a fully general tool, so the control is only as good as each approval decision.

B · Why this does not meet the full requirement

Tell the subagent in its prompt to query only the clause library and never modify any data

A prompt instruction is advisory; the tool still accepts any SQL the model writes.

The prompt already implied the purpose, and the agent strayed anyway. An instruction shapes likely behaviour but places no limit on what run_sql will execute, so a confused or manipulated turn can repeat the incident.

C · Why this meets the requirement

Replace run_sql with get_clause, which takes a clause ID and runs one fixed read-only query

The tool's contract allows only the intended read, so misuse cannot be expressed at all.

get_clause(clause_id) runs a parameterized SELECT on the clause table with a read-only connection. The subagent can still do its whole job, and neither salary queries nor updates can be expressed, whatever the model writes. Shared database code can remain behind the narrow interface.

D · Why this does not meet the full requirement

Keep run_sql but reject any statement that contains UPDATE, DELETE, or the name of the salary table

Keyword filters miss aliases, views and other tables, and every gap is a working bypass.

Blocking keywords catches the obvious cases but misses others: a different sensitive table, a view over salaries, or a write phrased another way. Each gap is a working bypass, and the list needs constant upkeep.

Check your understanding · explain the design in your own words

The agent's SQL tool lets it do far more than read clauses. How do you stop that for real?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Reuse implementation without exposing unrestricted tools Study: MCP security: architect-grade answers

Study: Reuse implementation without exposing unrestricted tools Study: MCP security: architect-grade answers Least privilege Allowlist Confused deputy

Return to Q316's situation and clause-by-clause explanation

Q317. A hospital records pipeline has extract_lab_results ('Extracts lab result data') and extract_discharge_summary ('Extracts discharge summary data'). Discharge summaries include a table of lab values, and the agent sends about 70% of them to the lab tool, losing diagnoses and follow-up instructions. Which rewrite of the definitions most directly fixes the routing? A) Discharge tool: full summaries, including embedded lab tables. Lab tool: standalone lab reports only; use the discharge tool for summaries B) Discharge tool: a much longer description listing all 22 output fields it extracts, with the lab tool's description left exactly as it is C) Lab tool: 'Extracts lab values, units, reference ranges and flags from any document that contains a lab table.' D) Lab tool: 'Extracts lab result data. IMPORTANT: this is the preferred tool for any clinical document.'

Understand Q317 · the situation and each part of the question

The real situation behind this question

A hospital digitizes discharge paperwork so follow-up clinics receive structured data. Discharge summaries always include a table of recent lab values. The agent sees that table and routes most summaries to the lab extractor, which returns tidy lab rows and silently drops diagnoses, medications and follow-up instructions. Clinics have started missing appointments that were written only in the dropped sections.

Read the question in parts

Part 1 · Discharge summaries include a table of lab values

The overlap is structural; words inside the document point at the wrong tool unless the descriptions say otherwise.

Part 2 · most directly fixes the routing

The fix belongs where the model chooses, in the tool definitions, rather than in a separate control layer.

What you are being asked to decide

Discharge summaries contain lab tables and get sent to the lab tool. Which description change fixes that? Write tool boundaries that route by document identity, not by words inside it.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Overlap casea document of one type that contains content typical of another tool's domain
Document identitywhat the document is (a discharge summary), as opposed to what sections it contains
Negative boundarya statement of what a tool is not for, naming the tool to use instead

Study: Document identity versus words inside it Study: What the descriptions should say Tool description Tool selection Structured outputs

Understand Q317's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Overlap cases misroute when descriptions describe content but not boundaries. Assign the overlap case to one tool, say so in both definitions, and route by what the document is, not by the words inside it. Emphasis and field lists do not draw the line.

Connect the design to the situation

A discharge summary with an embedded lab table goes to the lab extractor.

Best answer: A. The misrouting comes from overlap: discharge summaries contain lab tables, and nothing tells the model which tool owns that case. Stating that the discharge tool covers summaries including their lab tables, and that the lab tool is for standalone reports only, resolves the overlap where the model decides. Listing output fields adds detail without drawing the boundary.

What each choice would do here

A · Why this meets the requirement

Discharge tool: full summaries, including embedded lab tables. Lab tool: standalone lab reports only; use the discharge tool for summaries

It assigns the overlap case to one tool explicitly and tells the other tool to step aside for it.

The discharge description claims full summaries including their lab tables, and the lab description limits itself to standalone lab reports and says to use extract_discharge_summary for summaries. The overlap case now has one owner, stated in both definitions the model reads.

B · Why this does not meet the full requirement

Discharge tool: a much longer description listing all 22 output fields it extracts, with the lab tool's description left exactly as it is

More field detail describes output, but the routing question is which document type each tool accepts.

Listing 22 output fields makes the discharge tool's output clearer, and it may help a little. But it never says that summaries with lab tables belong here, and the unchanged lab tool still looks like a match for any lab table.

C · Why this does not meet the full requirement

Lab tool: 'Extracts lab values, units, reference ranges and flags from any document that contains a lab table.'

Claiming any document with a lab table widens the overlap and guarantees discharge summaries go to the wrong tool.

This description is detailed and accurate about lab extraction, but 'any document that contains a lab table' explicitly claims discharge summaries. It turns an ambiguous overlap into a defined rule that routes them wrongly.

D · Why this does not meet the full requirement

Lab tool: 'Extracts lab result data. IMPORTANT: this is the preferred tool for any clinical document.'

Shouting priority pulls even more documents to the lab tool and still says nothing about discharge summaries.

An IMPORTANT marker tells the model to favour the lab tool broadly, so even more discharge summaries go there. It adds emphasis without information about which document type belongs to which tool, making the misrouting worse.

Check your understanding · explain the design in your own words

Discharge summaries contain lab tables and get sent to the lab tool. Which description change fixes that?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Document identity versus words inside it Study: What the descriptions should say

Study: Document identity versus words inside it Study: What the descriptions should say Tool description Tool selection Structured outputs

Return to Q317's situation and clause-by-clause explanation

Drill · Context & Reliability (Q318–Q339)

Q318. A platform engineer has Claude Code migrating 40 microservices from a homegrown logger to structured logging. For the first dozen services its notes named the exact wrapper functions it replaced, such as logCtx() and emitAudit(). By service 26 the notes read "this service uses the usual logger setup," and the session has already been compacted twice. Fourteen services remain. Which plan restores precise work for those fourteen? A) Write the replacements found so far to a migration notes file, then hand each remaining service to a subagent that reads that file and returns a short structured list of the functions it changed. B) Clear the conversation and have one fresh session rescan all 40 services from the beginning so the rebuilt context contains only the essential names. C) Run the compaction a third time with a focus instruction to preserve every function name, then carry on with the remaining services in the same session. D) Add a line to the prompt telling the agent to always cite exact function names and file paths, and resume the migration in the current session.

Understand Q318 · the situation and each part of the question

The real situation behind this question

A platform team is replacing a homegrown logging library across 40 microservices before an observability deadline. The engineer running Claude Code noticed that the early migration notes were precise and useful for reviewers, but the later notes turned generic. Reviewers now cannot tell which wrappers were replaced in the later services. With fourteen services left, the engineer needs the rest of the migration to be as precise as the first dozen without redoing the whole job.

Read the question in parts

Part 1 · compacted twice

Compaction has already been tried and failed, so another round of it is not a new remedy.

Part 2 · Fourteen services remain

The remaining work is substantial and will add more file reads, so the plan must stop further crowding.

What you are being asked to decide

The main session has gone vague after lots of file reading. How do you finish the job precisely? Recognising context degradation and choosing isolation plus persisted notes over re-prompting or re-compacting.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Context isolationEach remaining service is read inside its own subagent context, so its file dumps never reach the main session.
ScratchpadThe migration notes file holding every replacement found so far, readable by any new context.

Study: Context management Study: Scratchpads & crash-recovery manifests Context isolation Scratchpad Subagent

Understand Q318's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Vague references after long exploration mean the specifics were crowded out, and repeated compaction cannot restore them. Persist what you know to a file, then isolate the remaining exploration in subagents that return compact summaries, so the main context never fills with raw reads again.

Connect the design to the situation

A logger migration across 40 services whose notes drift from exact function names to generic phrases.

Best answer: A. The vague notes show that earlier specifics have been pushed out by accumulated file reads, and compaction has already failed twice to bring them back. Saving what is known to a file and giving each remaining service its own subagent context keeps the noisy reads out of the main session. A prompt demanding exact names does nothing, because the names are no longer in context to cite.

What each choice would do here

A · Why this meets the requirement

Write the replacements found so far to a migration notes file, then hand each remaining service to a subagent that reads that file and returns a short structured list of the functions it changed.

Correct: it persists the specifics and stops new exploration from crowding the main context.

Persisting the replacements to a file protects the finished work from further loss. Each subagent reads that file, does its noisy exploration in a separate context, and returns only a compact list, so the main session stays sharp for the final fourteen services.

B · Why this does not meet the full requirement

Clear the conversation and have one fresh session rescan all 40 services from the beginning so the rebuilt context contains only the essential names.

A full rescan throws away 26 services of work and will degrade the same way again.

Starting over costs every service already migrated and relies on one session holding 40 services of reads, which is exactly the load that degraded the first attempt. The same drift will return partway through.

C · Why this does not meet the full requirement

Run the compaction a third time with a focus instruction to preserve every function name, then carry on with the remaining services in the same session.

A third compaction summarises an already blurred history; it cannot recover names that are gone.

A focus instruction helps compaction decide what to keep from what is currently present, but the specific names were already blurred in earlier rounds. Compacting again produces a shorter summary of vague material and the next service reads push it out once more.

D · Why this does not meet the full requirement

Add a line to the prompt telling the agent to always cite exact function names and file paths, and resume the migration in the current session.

An instruction to be specific cannot make the model cite details that are no longer in its context.

The model is not choosing to be vague; it no longer holds the specifics. Telling it to cite exact names may even encourage invented names, because the instruction pressures it to produce detail it cannot see.

Check your understanding · explain the design in your own words

The main session has gone vague after lots of file reading. How do you finish the job precisely?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Context management Study: Scratchpads & crash-recovery manifests

Study: Context management Study: Scratchpads & crash-recovery manifests Context isolation Scratchpad Subagent

Return to Q318's situation and clause-by-clause explanation

Q319. An agent generates migration scripts for 30 warehouse database tables, one table at a time. After each table it summarises the schema in a sentence or two and drops the raw DDL to save room. While handling table 9 it saw that the column cust_ref had been renamed to customer_id everywhere. The script it wrote for table 24 still joins on cust_ref. Which design change would have prevented this? A) Keep a small ledger of cross-table facts, such as renames and type changes, outside the summaries and include it unchanged in every step. B) Switch to a model with a larger context window so the raw DDL for every table fits and nothing has to be summarised at all. C) Lower the sampling temperature for the script-writing step so the agent reproduces column names more consistently across tables. D) Before each new table, have the agent reread its own summary of table 9 so the earlier change stays fresh in its working context.

Understand Q319 · the situation and each part of the question

The real situation behind this question

A data team is moving a warehouse to a new schema and uses an agent to write migration scripts for 30 tables. To stay within budget the agent keeps only a brief summary of each finished table. Late in the run a script joined on a column that no longer exists, and the migration failed in staging. The team wants to know what design would keep a change learned early from disappearing by the time later tables depend on it.

Read the question in parts

Part 1 · summarises the schema in a sentence or two

Each table's detail is squeezed into a tiny summary, so small but important changes can be dropped.

Part 2 · renamed to `customer_id` everywhere

The fact affects every later table, which makes it cross-cutting rather than local to table 9.

What you are being asked to decide

Why did a rename seen earlier vanish, and what structure keeps such facts safe? Identifying summary loss as the failure and choosing a durable facts block as the fix.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CompactionReplacing detailed table history with a short summary, which can drop a single rename.
ScratchpadThe rename ledger that sits outside the summaries and is carried whole into each step.

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests Compaction Scratchpad Context budget

Understand Q319's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Summaries drop detail, and a rename is exactly the kind of small detail that disappears. Anything later steps depend on must be kept outside the summarised history in a durable facts block included whole at every step.

Connect the design to the situation

A table-by-table migration where a renamed column reappears under its old name fifteen tables later.

Best answer: A. The rename was compressed away when table 9's detail was replaced by a short summary, so later steps only saw the older name. A ledger of cross-cutting facts that is never summarised survives every step. Rereading the table 9 summary does not help, because that summary is exactly where the rename was lost.

What each choice would do here

A · Why this meets the requirement

Keep a small ledger of cross-table facts, such as renames and type changes, outside the summaries and include it unchanged in every step.

Correct: facts that affect later steps are kept verbatim and never pass through a lossy summary.

A ledger of renames and type changes is small, is appended to the moment a cross-table fact appears, and is included unchanged in every step. The table 24 step would have seen that `cust_ref` is now `customer_id` and written the correct join.

B · Why this does not meet the full requirement

Switch to a model with a larger context window so the raw DDL for every table fits and nothing has to be summarised at all.

A bigger window postpones summarising but does not change a design that drops facts when it does summarise.

A larger window would let more DDL stay, but the team would still summarise once the run grew past it. The underlying design, where every fact passes through a lossy summary, stays unchanged and the rename could vanish later.

C · Why this does not meet the full requirement

Lower the sampling temperature for the script-writing step so the agent reproduces column names more consistently across tables.

Temperature shapes wording variety, not whether a fact is still present in the context.

A lower temperature makes the model choose its most likely tokens, but the most likely token is the old name it still holds. Consistency is not the problem; the missing rename is.

D · Why this does not meet the full requirement

Before each new table, have the agent reread its own summary of table 9 so the earlier change stays fresh in its working context.

The table 9 summary is the lossy artifact; rereading it repeats the loss rather than repairing it.

The summary of table 9 is where the rename was reduced to something like minor schema tweaks. Reading that summary again puts the same incomplete text back into context, so the agent keeps using the older column name.

Check your understanding · explain the design in your own words

Why did a rename seen earlier vanish, and what structure keeps such facts safe?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests Compaction Scratchpad Context budget

Return to Q319's situation and clause-by-clause explanation

Q320. A security team used an agent to condense 200 internal policy documents into a two-page answer for a customer's vendor questionnaire. One line reads "customer data is encrypted at rest with customer-managed keys." A year later an auditor asks which policy supports that line and whether it still applies after the key-management redesign. Which output design would have made that question a quick lookup? A) Run a second model pass over the answer before sending it to confirm each sentence against the policy set at that time. B) Attach a bibliography at the end of the answer listing all 200 policy documents that were given to the agent as input. C) Store each claim as a record naming the source document, section, quoted excerpt and retrieval date alongside the sentence. D) Save every generated answer in a document store with version history so each line can be traced to the date it was produced.

Understand Q320 · the situation and each part of the question

The real situation behind this question

A software vendor answers security questionnaires from enterprise customers, and the security team used an agent to summarise hundreds of policy documents into short answers. Since then the company redesigned its key management. An auditor now wants proof for one encryption claim and wants to know if it still holds. Without knowing which policy text produced the sentence, the team faces rereading all 200 documents to answer one question.

Read the question in parts

Part 1 · which policy supports that line

The auditor wants the specific source of a single sentence, not the whole input set.

Part 2 · after the key-management redesign

The source may have changed, so the team needs to know exactly which document to recheck.

What you are being asked to decide

How do you make a synthesised claim checkable a year later? Choosing per-claim provenance over document-level history or one-time verification.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
ProvenanceThe record tying the encryption sentence to its exact policy section and date read.
Audit trailThe evidence an auditor follows to confirm a claim without redoing the whole synthesis.

Study: Provenance — claim-source mappings through synthesis Provenance Audit trail

Understand Q320's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Synthesis condenses many sources into sentences and drops which source produced which sentence unless you keep it. Store claim-to-source records with location and date so later verification is a lookup, not a re-audit.

Connect the design to the situation

A condensed questionnaire answer whose encryption claim must be traced after a redesign.

Best answer: C. The auditor needs the link from one sentence to the specific evidence behind it, and only a per-claim provenance record carries that. Version history dates the sentence but never names its source, and a list of all inputs does not say which one supports which claim.

What each choice would do here

A · Why this does not meet the full requirement

Run a second model pass over the answer before sending it to confirm each sentence against the policy set at that time.

A pre-send check helps accuracy on the day, but leaves no trace for the auditor a year later.

A verification pass may have caught errors when the answer was written, but it leaves no lasting link. A year later, after the redesign, its result is stale and does not show where to look.

B · Why this does not meet the full requirement

Attach a bibliography at the end of the answer listing all 200 policy documents that were given to the agent as input.

Listing every input does not tell anyone which document supports this particular claim.

A bibliography of 200 inputs shows what the agent was given, not what it used. The auditor learns nothing about which of the 200 supports the encryption line, so the search effort is unchanged.

C · Why this meets the requirement

Store each claim as a record naming the source document, section, quoted excerpt and retrieval date alongside the sentence.

Correct: each claim carries its own source, location and date, so re-verification is targeted.

A claim record pointing to the key-management policy, section and excerpt, with the date it was read, turns the audit into opening one section and comparing it with the redesigned policy. Every other claim can be rechecked the same way.

D · Why this does not meet the full requirement

Save every generated answer in a document store with version history so each line can be traced to the date it was produced.

Version history shows when a line was written, not which policy it came from.

A version store records that the answer existed on a given date and who changed it. It holds no link between the encryption sentence and any policy section, so the team would still have to search all 200 documents.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

One claim with its provenance

JSON
{
  "claim": "Customer data is encrypted at rest with customer-managed keys.",
  "sources": [
    {
      "doc_id": "POL-KMS-004",
      "section": "3.2 Key custody",
      "excerpt": "Tenant data stores use keys held in the customer key vault...",
      "retrieved_at": "2025-08-14"
    }
  ]
}

The auditor opens POL-KMS-004 section 3.2 and compares it with the redesigned policy; no other document needs rereading.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you make a synthesised claim checkable a year later?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Provenance — claim-source mappings through synthesis

Study: Provenance — claim-source mappings through synthesis Provenance Audit trail

Return to Q320's situation and clause-by-clause explanation

Q321. A trip-planning coordinator sends a hotel-availability subagent to check five cities. After finishing three cities, the subagent's upstream booking service starts returning rate-limit responses with a retry-after header. What should the subagent send back so the coordinator can make the best recovery decision? A) Throw the rate-limit exception up to the coordinator's top-level handler, which ends the whole trip plan so that no partial itinerary is produced. B) Return an error result stating a rate-limit failure, the three completed cities with results, the two pending cities, and the retry-after value. C) Retry internally with backoff until its budget is spent, then return one short message stating that hotel search is currently unavailable. D) Return the three finished cities as a normal success result so the itinerary can be built, leaving the other two cities out of the reply.

Understand Q321 · the situation and each part of the question

The real situation behind this question

A travel company's planning assistant builds multi-city itineraries by sending subagents to check flights, hotels and events. During a busy weekend the hotel provider throttles requests partway through a five-city search. The coordinator could wait and retry, try another provider for the missing cities, or present the itinerary with gaps flagged. Which of those is possible depends entirely on what the hotel subagent reports when it runs into the limit.

Read the question in parts

Part 1 · After finishing three cities

There is real partial work that should reach the coordinator rather than be discarded.

Part 2 · a retry-after header

The failure is transient and comes with timing, which is recovery information worth passing up.

What you are being asked to decide

What should a partly failed subagent report so the coordinator can recover? Designing failure payloads that preserve partial work and recovery options.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Error taxonomyLabelling the failure as a transient rate limit so the coordinator knows waiting can work.
Graceful degradationUsing the three finished cities now while the two pending ones are retried or rerouted.

Study: Error taxonomy — the harness branches on these Study: Resilience patterns Error taxonomy Graceful degradation tool_result

Understand Q321's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Recovery happens at the coordinator, so the subagent's job is to report failure in a form the coordinator can reason about. Return the failure type, what completed with its results, what is pending and any retry hint, never a silent success or a bare error.

Connect the design to the situation

A hotel lookup that completes three of five cities before hitting rate limits.

Best answer: B. The coordinator can only choose between waiting, rerouting or continuing with partial data if it can see the failure type, what finished, what is pending and when retry is sensible. A generic unavailable message removes all of that, and reporting three cities as full success hides the gap entirely.

What each choice would do here

A · Why this does not meet the full requirement

Throw the rate-limit exception up to the coordinator's top-level handler, which ends the whole trip plan so that no partial itinerary is produced.

Ending the whole plan throws away three good city results over a temporary problem.

Escalating the exception to a handler that ends everything turns a temporary limit into a total failure. The traveller gets nothing, even though three cities of results were ready and the other two might have succeeded a minute later.

B · Why this meets the requirement

Return an error result stating a rate-limit failure, the three completed cities with results, the two pending cities, and the retry-after value.

Correct: type, partial results, pending work and retry timing let the coordinator choose well.

With a structured error the coordinator can present three cities now, schedule a retry after the given delay, or send the two pending cities to another provider. Every recovery path stays open because nothing was hidden.

C · Why this does not meet the full requirement

Retry internally with backoff until its budget is spent, then return one short message stating that hotel search is currently unavailable.

A generic unavailable message discards the partial results and the retry timing.

Internal retries may be sensible, but collapsing the outcome into one unavailable message discards the three good cities and the retry timing. The coordinator can only fail the whole hotel step or guess.

D · Why this does not meet the full requirement

Return the three finished cities as a normal success result so the itinerary can be built, leaving the other two cities out of the reply.

Calling a partial result a success hides the two missing cities from the coordinator.

The coordinator believes all five cities were searched and builds an itinerary with two cities lacking hotels and no explanation. Nobody retries the missing cities, because nothing reported them as missing.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Structured partial-failure result

JSON
{
  "status": "partial_failure",
  "error_type": "rate_limited",
  "retryable": true,
  "retry_after_seconds": 30,
  "completed": {
    "Lisbon": [
      "Hotel Almada"
    ],
    "Porto": [
      "Casa Ribeira"
    ],
    "Madrid": [
      "Hostal Sol"
    ]
  },
  "pending": [
    "Seville",
    "Valencia"
  ],
  "alternatives": [
    "retry after delay",
    "query secondary provider for pending cities"
  ]
}

The coordinator can use three cities immediately and decide how to handle the two pending ones.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

What should a partly failed subagent report so the coordinator can recover?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Error taxonomy — the harness branches on these Study: Resilience patterns

Study: Error taxonomy — the harness branches on these Study: Resilience patterns Error taxonomy Graceful degradation tool_result

Return to Q321's situation and clause-by-clause explanation

Q322. A research assistant is preparing evidence for a city infrastructure committee on a concrete additive. A peer-reviewed 2019 lab study on 40 test slabs reports a 30% reduction in cracking. A 2025 white paper funded by the manufacturer, covering 900 slabs in real roads, reports 8%. Which statement describes the synthesis agent's job exactly? A) Present both results, each with its source, date, review status, funding, sample size and setting, and note what may explain the gap. B) Weight the two figures by their sample sizes and report the combined estimate of roughly 9% as the best overall effect. C) Treat the peer-reviewed lab study as authoritative because it passed review, and mention the white paper only as an unreviewed claim. D) Leave both results out because they disagree, and tell the committee that no conclusion is possible until a third study is published.

Understand Q322 · the situation and each part of the question

The real situation behind this question

A city's infrastructure committee must decide whether to specify a new additive in road concrete, a choice worth millions over a decade. The research assistant found two relevant sources that disagree sharply. One is a small, independently reviewed lab study; the other is large and real-world but paid for by the seller. Committee members, not the assistant, are accountable for the call, so they need the evidence in a form they can judge.

Read the question in parts

Part 1 · funded by the manufacturer

Funding is a provenance attribute the committee should see, not grounds for silent exclusion.

Part 2 · covering 900 slabs in real roads

Setting and sample size differ between the studies and may explain the conflicting figures.

What you are being asked to decide

Two credible studies disagree. What does the synthesis owe its readers? Preserving conflicting evidence with provenance instead of resolving it on the reader’s behalf.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
ProvenanceSource, date, review status, funding and setting attached to each crack-reduction figure.
Graceful degradationDelivering the evidence with its uncertainty visible instead of withholding everything.

Study: Provenance — claim-source mappings through synthesis Provenance Graceful degradation

Understand Q322's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Conflicting credible sources are evidence, not noise to clean up. Keep both values and attach source, date, review status, funding, sample size and setting so the accountable readers can weigh them.

Connect the design to the situation

A small reviewed lab study and a large vendor-funded field report disagree on crack reduction.

Best answer: A. Lab slabs and real roads, independent review and manufacturer funding are differences the committee must weigh, and only presenting both with that context lets it do so. Weighting by sample size manufactures a single number that neither study reported and hides why they differ.

What each choice would do here

A · Why this meets the requirement

Present both results, each with its source, date, review status, funding, sample size and setting, and note what may explain the gap.

Correct: both figures and the attributes that explain them reach the committee.

Both figures appear with their attributes, and a note points out that lab slabs and in-service roads face different loads and curing. The committee can decide how much weight to give each and may request an independent field trial.

B · Why this does not meet the full requirement

Weight the two figures by their sample sizes and report the combined estimate of roughly 9% as the best overall effect.

A pooled figure hides the lab-versus-road difference that likely explains the gap.

Pooling by sample size produces a number dominated by the white paper and reported by neither study. The difference between lab and road conditions disappears, so the committee cannot see the question it most needs to discuss.

C · Why this does not meet the full requirement

Treat the peer-reviewed lab study as authoritative because it passed review, and mention the white paper only as an unreviewed claim.

Review status is one factor to show, not a rule that entitles the agent to discard field data.

The committee sees 30% presented as settled and 8% dismissed. A real-world result from 900 slabs may be the more relevant one for roads, and the agent has made a judgment that belongs to the committee.

D · Why this does not meet the full requirement

Leave both results out because they disagree, and tell the committee that no conclusion is possible until a third study is published.

Dropping both leaves the committee with nothing even though two informative studies exist.

Withholding both studies because they disagree gives the committee nothing to work with. A disagreement between credible sources is itself useful information, and the decision cannot wait indefinitely for a third study.

Check your understanding · explain the design in your own words

Two credible studies disagree. What does the synthesis owe its readers?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Provenance — claim-source mappings through synthesis

Study: Provenance — claim-source mappings through synthesis Provenance Graceful degradation

Return to Q322's situation and clause-by-clause explanation

Q323. A labor-market dashboard agent finds that 34% of tech job postings were fully remote in a survey fielded in early 2022, and 22% in a different survey from late 2025. The current design marks every such mismatch as a contradiction and holds the report until a human analyst resolves it. What is wrong with this design? A) Nothing important; escalating every mismatch to an analyst is the safest way to keep contradictory figures out of the published dashboard. B) The design should resolve such mismatches automatically by keeping the most recent figure, since it reflects current market conditions. C) The design should publish the midpoint of the two figures, 28%, with a note that the underlying surveys disagreed somewhat. D) The figures come from different periods, so they may describe a trend, and should be shown together with their sources and dates.

Understand Q323 · the situation and each part of the question

The real situation behind this question

A staffing firm publishes a monthly labor-market dashboard for clients deciding on office policy. Its agent pulls figures from many public surveys, and whenever two numbers on the same topic differ it pauses the report for an analyst. Analysts are swamped, and many of the flagged cases involve surveys from different years. Clients mainly want to know which way the market is moving, and the dates are exactly what show that.

Read the question in parts

Part 1 · holds the report until a human analyst resolves it

The current design treats any difference as a conflict needing a person, which is costly.

Part 2 · early 2022

The first figure has a date, which suggests the two numbers measure different moments.

What you are being asked to decide

Are two figures from different years really in conflict? Separating temporal difference from true conflict and keeping dates attached to values.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
ProvenanceThe survey name and fielding date carried with each remote-work percentage.
EscalationRouting a mismatch to a human, which is wasted here because the figures do not actually conflict.

Study: Provenance — claim-source mappings through synthesis Provenance Escalation Human-in-the-loop

Understand Q323's answer · full reasoning and worked examples
Claude synthesized · the point, straight

A difference between values from different periods is often a trend, not a conflict. Attach source and date to each value and publish both, reserving conflict handling for figures that describe the same thing at the same time.

Connect the design to the situation

Remote-job share of 34% in 2022 and 22% in 2025 flagged as a contradiction.

Best answer: D. Two readings years apart are different moments, and the change between them may be the story. Treating them as a contradiction needing resolution sends work to an analyst for nothing and pushes toward choosing one; dating both lets the reader see the decline.

What each choice would do here

A · Why this does not meet the full requirement

Nothing important; escalating every mismatch to an analyst is the safest way to keep contradictory figures out of the published dashboard.

Escalation spends analyst time on what is not a contradiction and delays the dashboard.

Sending a dated pair of figures to an analyst produces the answer the dates already gave: they are two points in time. The report is delayed and analysts spend effort on cases that need no judgment at all.

B · Why this does not meet the full requirement

The design should resolve such mismatches automatically by keeping the most recent figure, since it reflects current market conditions.

Keeping only the latest figure erases the history that shows the decline.

Automatically keeping the newest figure is fast, but clients lose the earlier data point and the trend with it. It also teaches the system to drop data whenever values differ, which is wrong for truly simultaneous conflicts.

C · Why this does not meet the full requirement

The design should publish the midpoint of the two figures, 28%, with a note that the underlying surveys disagreed somewhat.

A midpoint invents a value that no survey measured at any time.

A 28% midpoint corresponds to no survey and no date. Clients reading it would think the remote share is fairly stable, the opposite of what the two surveys show together.

D · Why this meets the requirement

The figures come from different periods, so they may describe a trend, and should be shown together with their sources and dates.

Correct: different periods can both be right, and dates make the trend visible.

Presenting 34% for early 2022 and 22% for late 2025, each with its survey, shows clients a clear decline in remote postings. No human review is needed and no figure is discarded or altered.

Check your understanding · explain the design in your own words

Are two figures from different years really in conflict?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Provenance — claim-source mappings through synthesis

Study: Provenance — claim-source mappings through synthesis Provenance Escalation Human-in-the-loop

Return to Q323's situation and clause-by-clause explanation

Q324. A pull-request review pipeline runs a license-compliance subagent alongside style and security checks. The license scanner's API token had expired, so its calls returned 401 Unauthorized. The subagent caught the error and returned {"violations": []}, and the pipeline approved a PR that added a library under an incompatible license. Where is the defect? A) In the subagent's retry policy, which should have retried the 401 with exponential backoff before giving up and returning results. B) In the orchestration design, which should halt all review for every repository as soon as any single check fails for any reason. C) In the pipeline, which should refuse to approve any PR unless every check subagent reports at least one finding for a reviewer to look at. D) In the subagent, which turned an authorization failure into a clean-looking empty result, so nothing upstream knew the scan never ran.

Understand Q324 · the situation and each part of the question

The real situation behind this question

An engineering organisation relies on automated checks to approve pull requests quickly. Legal requires that no library with an incompatible license ships. When the license scanner's token quietly expired, every scan came back empty, and for several days the pipeline treated every PR as license-clean. One PR introduced a copyleft dependency that went to production. The team now needs to pin down which piece of the design let a failure look like a pass.

Read the question in parts

Part 1 · returned `{"violations": []}`

The subagent reported a normal clean result, the same shape a genuinely clean PR produces.

Part 2 · returned 401 Unauthorized

The scan was refused for credentials, so no licensing information was ever gathered.

What you are being asked to decide

A check failed but reported clean. Which component is at fault? Locating the silent error suppression as the root cause instead of adding the wrong compensating control.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Error taxonomyClassifying the 401 as an access failure, which is neither transient nor a valid empty result.
tool_resultWhat the subagent handed back, which should have signalled an error instead of empty success.

Study: Error taxonomy — the harness branches on these Error taxonomy Graceful degradation tool_result

Understand Q324's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Returning empty success for a failed call makes a missed check indistinguishable from a passed one. Subagents must report access failures as errors with their type so the coordinator can block, reroute or flag the gap.

Connect the design to the situation

An expired scanner token produces an empty violation list and an incorrectly approved PR.

Best answer: D. An empty violations list is a legitimate outcome for a clean PR, so the pipeline had no way to tell a clean scan from a scan that never happened. The subagent should have returned an access error so the pipeline could block, alert or mark the check incomplete.

What each choice would do here

A · Why this does not meet the full requirement

In the subagent's retry policy, which should have retried the 401 with exponential backoff before giving up and returning results.

An expired token will not fix itself, so retries only delay the same failure.

A 401 means the credentials are invalid, and backoff does not renew a token. After the retries the subagent would still face the question of how to report the failure, and it would still have reported an empty list.

B · Why this does not meet the full requirement

In the orchestration design, which should halt all review for every repository as soon as any single check fails for any reason.

Stopping all review everywhere over one failed check is far out of proportion.

A global halt on any failure would stop every team's reviews for one expired token. The real need is for the failure to be visible so this PR is held, not for unrelated work to stop.

C · Why this does not meet the full requirement

In the pipeline, which should refuse to approve any PR unless every check subagent reports at least one finding for a reviewer to look at.

Clean PRs legitimately have no findings, so demanding findings would block good changes.

Requiring at least one finding would fail every genuinely clean PR and push reviewers to ignore the rule. It also would not reveal that the license check did not run, because other checks could still supply findings.

D · Why this meets the requirement

In the subagent, which turned an authorization failure into a clean-looking empty result, so nothing upstream knew the scan never ran.

Correct: masking the error as empty success is what hid the missed scan.

If the subagent had returned an access-failure result, the pipeline could mark the license check as not run, hold the PR and alert whoever owns the scanner token. The incompatible library would never have been approved.

Check your understanding · explain the design in your own words

A check failed but reported clean. Which component is at fault?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Error taxonomy — the harness branches on these

Study: Error taxonomy — the harness branches on these Error taxonomy Graceful degradation tool_result

Return to Q324's situation and clause-by-clause explanation

Q325. A recruiting agent runs two subagents. One searches the internal candidate database for "Rust and FPGA experience, based in Lisbon"; the query completes normally and returns zero candidates. The other asks the interviewer-calendar service for open slots and receives a 503 Service Unavailable. The harness currently treats both as "no data" and retries each three times. Which handling rule is right? A) Treat both as valid empty answers and move on, reporting that there are no suitable candidates and no open interview slots. B) Retry both with a longer exponential backoff, since either could succeed on a later attempt if the systems are given more time. C) Accept the zero-candidate result as a real answer and note it, and retry the calendar call as a transient service failure. D) Send both straight to the recruiting operations team for manual investigation and remove the automatic retries entirely.

Understand Q325 · the situation and each part of the question

The real situation behind this question

A recruiting team uses an agent to shortlist candidates and book interviews for niche roles. Hiring managers have complained that the agent reports "no candidates" and "no availability" in the same breath, and that runs take long because every empty response is retried. The harness does not distinguish a search that found nothing from a service that failed to answer, so both waste time and one of them gives a false answer.

Read the question in parts

Part 1 · the query completes normally and returns zero candidates

The search ran successfully; zero is its real answer, not an error.

Part 2 · receives a 503 Service Unavailable

The calendar service failed temporarily, so no answer about slots was obtained.

What you are being asked to decide

One call found nothing; the other failed. Should they be handled alike? Branching on outcome type: valid empty versus transient failure.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Error taxonomySeparating a completed empty search from a 503, which is a transient failure.
BackpressureRespecting an overloaded service by retrying with delay instead of hammering it.

Study: Error taxonomy — the harness branches on these Study: Resilience patterns Error taxonomy Backpressure

Understand Q325's answer · full reasoning and worked examples
Claude synthesized · the point, straight

An empty result from a query that ran is an answer; a 503 is a failure that may pass. Branch on the outcome type: accept and record the empty answer, retry the transient failure, and never report one as the other.

Connect the design to the situation

A zero-hit candidate search and a 503 calendar call treated identically.

Best answer: C. A completed search with no matches is a finding; retrying it just repeats the same answer. A 503 is a transient failure where retrying makes sense. Treating both as the same outcome either wastes calls or hides that the calendar was never read.

What each choice would do here

A · Why this does not meet the full requirement

Treat both as valid empty answers and move on, reporting that there are no suitable candidates and no open interview slots.

Calling the 503 an empty answer falsely reports that no interview slots exist.

The hiring manager would be told that no interview slots exist when the calendar was simply unreachable. Interviews would not be booked even though slots may be available, and nobody would know to try again.

B · Why this does not meet the full requirement

Retry both with a longer exponential backoff, since either could succeed on a later attempt if the systems are given more time.

Retrying the finished search only reproduces zero results and wastes calls.

Longer backoff on the candidate search changes nothing; the database will return zero again because no one matches. The calendar retry is reasonable, but applying it to both adds delay and hides the difference between the outcomes.

C · Why this meets the requirement

Accept the zero-candidate result as a real answer and note it, and retry the calendar call as a transient service failure.

Correct: each outcome gets the handling its type calls for.

The shortlist step records that no Lisbon candidates with Rust and FPGA experience exist, perhaps suggesting a wider search, while the calendar call is retried with backoff. The manager gets an accurate picture of both results.

D · Why this does not meet the full requirement

Send both straight to the recruiting operations team for manual investigation and remove the automatic retries entirely.

Manual investigation of both is overkill for a routine empty search and a transient outage.

Operations staff would spend time confirming that a niche search found no one and waiting for a calendar outage to pass. Neither case needs human judgment; each has a standard automatic response.

Check your understanding · explain the design in your own words

One call found nothing; the other failed. Should they be handled alike?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Error taxonomy — the harness branches on these Study: Resilience patterns

Study: Error taxonomy — the harness branches on these Study: Resilience patterns Error taxonomy Backpressure

Return to Q325's situation and clause-by-clause explanation

Q326. An insurer auto-approves repair estimates from photo analysis when model confidence is at least 0.88, a threshold calibrated a year ago on photos of sedans and SUVs. A new fleet client sends photos of electric cargo vans taken with fisheye dashcams, and the estimates come back at 0.93 confidence. Which statement is exactly right? A) Raising the threshold for every vehicle type to 0.97 is the right response, since that removes most of the risk from unfamiliar photos. B) A 0.93 score says little about accuracy on photos unlike the calibration set; stratified sampling of auto-approved estimates must measure it. C) Asking the model whether it is sure about the vehicle type, and approving only when it says yes, confirms that the score is trustworthy. D) The estimates can be approved, because the threshold was calibrated against labelled data and 0.93 clears it by a comfortable margin.

Understand Q326 · the situation and each part of the question

The real situation behind this question

An insurer saves adjusters many hours by letting a photo model approve routine repair estimates. The threshold was tuned against thousands of labelled claims, all from passenger cars photographed on phones. A new commercial client brings a fleet of electric vans with wide-angle dashcam photos. The estimates arrive with high confidence, and the claims manager must decide whether that number deserves the same trust it earned on passenger cars.

Read the question in parts

Part 1 · calibrated a year ago on photos of sedans and SUVs

The score was only shown to track accuracy for that population of vehicles and cameras.

Part 2 · taken with fisheye dashcams

The new input differs in vehicle and camera, outside what calibration covered.

What you are being asked to decide

Does a high confidence score mean anything on input the system has never seen? Knowing that calibration is distribution-bound and that only sampled human review measures real accuracy.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CalibrationEvidence that a score of 0.88 meant about that accuracy, measured only on sedan and SUV photos.
Stratified samplingReviewing random auto-approved estimates from each vehicle type and camera source, including the new vans.

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design Calibration Stratified sampling Human-in-the-loop

Understand Q326's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Confidence thresholds are only calibrated for the inputs they were measured on, and a model cannot certify its own confidence. Keep sampling auto-approved output by segment so new input types are measured by humans rather than trusted on their score.

Connect the design to the situation

Fisheye photos of cargo vans scoring 0.93 against a threshold calibrated on sedans and SUVs.

Best answer: B. Calibration only holds for the kind of input it was measured on, so a high score on fisheye van photos is unverified. Continued stratified sampling across vehicle types and camera sources reveals the true error rate. Asking the model to confirm itself is still self-reported confidence and adds no independent evidence.

What each choice would do here

A · Why this does not meet the full requirement

Raising the threshold for every vehicle type to 0.97 is the right response, since that removes most of the risk from unfamiliar photos.

A blanket higher threshold costs review on familiar cases and still trusts an unverified score.

Moving every vehicle type to 0.97 sends many accurate sedan estimates to adjusters and wastes the savings. The van estimates could still score above 0.97 while being wrong, because the score has not been checked on them.

B · Why this meets the requirement

A 0.93 score says little about accuracy on photos unlike the calibration set; stratified sampling of auto-approved estimates must measure it.

Correct: the score is unvalidated here, and only sampled human review measures the true error rate.

Sampling auto-approved estimates from each segment, including fleet vans and dashcam photos, gives adjusters labelled outcomes for the new population. If the error rate is high, the segment moves to review; if low, automation continues with evidence.

C · Why this does not meet the full requirement

Asking the model whether it is sure about the vehicle type, and approving only when it says yes, confirms that the score is trustworthy.

The model's own yes or no is still self-reported and cannot validate its score.

The model's answer to whether it is sure comes from the same system whose confidence is in question. A confident misreading of fisheye photos will produce a confident yes, so this adds a step without adding independent evidence.

D · Why this does not meet the full requirement

The estimates can be approved, because the threshold was calibrated against labelled data and 0.93 clears it by a comfortable margin.

Calibration on sedans says nothing about fisheye van photos, so clearing the bar proves little.

Approving on the score assumes that 0.93 means the same on van dashcam photos as on phone photos of sedans. Distortion and unfamiliar bodywork may make the model confidently wrong, and those errors would be paid out without anyone noticing.

Check your understanding · explain the design in your own words

Does a high confidence score mean anything on input the system has never seen?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design Calibration Stratified sampling Human-in-the-loop

Return to Q326's situation and clause-by-clause explanation

Q327. At a regional hospital, coders currently check every billing code an assistant proposes for outpatient, surgical and emergency encounters; its headline figure is 96% overall accuracy. Finance proposes letting codes it is highly confident about post unchecked. Compliance will sign off only with the right protections. Which two belong in that sign-off? (Select TWO.) A) Have the assistant state in words how certain it is about each code, and auto-accept only the codes it describes as certain. B) Measure accuracy against a labelled set separately for each encounter type and code field, and set confidence thresholds per field from that data. C) Keep a stratified random sample of automatically accepted codes under ongoing human review to track error rates by segment. D) Require the 96% overall accuracy figure to stay steady for two consecutive quarters before any reduction in review takes effect.

Understand Q327 · the situation and each part of the question

The real situation behind this question

A hospital's revenue team spends heavily on coders who check every code the assistant proposes. The assistant's overall accuracy looks strong, and finance wants to let high-confidence codes through unchecked. Compliance is worried: miscoded surgical or emergency encounters can trigger payer audits and penalties. The team needs safeguards that show where automation is actually safe and that keep catching problems once humans stop checking every code.

Read the question in parts

Part 1 · 96% overall accuracy

An aggregate figure that may hide encounter types or fields with much lower accuracy.

Part 2 · highly confident about

The plan relies on confidence scores that have not yet been calibrated per field.

What you are being asked to decide

What must be true before you trust high-confidence output without review? Choosing segment-level calibration and ongoing sampling over aggregate figures and self-assessment.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CalibrationChecking per field whether a given confidence score matches measured accuracy on labelled encounters.
Stratified samplingReviewing random auto-accepted codes from every encounter type after automation begins.

Study: Confidence calibration & stratified sampling (human-review routing) Calibration Stratified sampling Human-in-the-loop

Understand Q327's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Aggregate accuracy and self-reported certainty cannot justify removing review. Calibrate thresholds per field and segment on labelled data, then keep sampling automated output by segment so errors stay measured.

Connect the design to the situation

A billing-code assistant at 96% overall accuracy that finance wants to automate.

Best answer: BC. Per-segment, per-field measurement with thresholds calibrated on labelled data shows where automation is safe, and ongoing stratified sampling keeps measuring after launch. A steady overall figure can hide a weak encounter type, and verbal certainty is self-reported and uncalibrated.

What each choice would do here

A · Why this does not meet the full requirement

Have the assistant state in words how certain it is about each code, and auto-accept only the codes it describes as certain.

Verbal certainty is self-reported and has not been checked against outcomes.

The assistant's statement that it is certain comes from the same model whose accuracy is in question. Without comparison against labelled outcomes, certain may be wrong at a high rate for some encounter types, and nobody would measure it.

B · Why this does not meet the full requirement

Measure accuracy against a labelled set separately for each encounter type and code field, and set confidence thresholds per field from that data.

Correct: it shows where accuracy is high enough to automate and sets thresholds from evidence.

Measuring separately for outpatient, surgical and emergency encounters and for each code field shows where accuracy is high. Thresholds set per field from labelled data mean a score of 0.9 corresponds to measured accuracy, not a guess.

C · Why this does not meet the full requirement

Keep a stratified random sample of automatically accepted codes under ongoing human review to track error rates by segment.

Correct: sampling keeps measuring real error rates once humans stop checking everything.

After launch, a random sample from each encounter type is still checked by coders. If a new procedure or payer rule causes errors, the sampled error rate rises in that segment and the team can restore review before audits find the problem.

D · Why this does not meet the full requirement

Require the 96% overall accuracy figure to stay steady for two consecutive quarters before any reduction in review takes effect.

A steady aggregate can hide one encounter type that is consistently wrong.

Two quarters of a steady 96% says the mix has not shifted, not that each segment is safe. If emergency encounters run at 80%, a stable aggregate would keep hiding them while automation lets their errors through.

Check your understanding · explain the design in your own words

What must be true before you trust high-confidence output without review?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Confidence calibration & stratified sampling (human-review routing)

Study: Confidence calibration & stratified sampling (human-review routing) Calibration Stratified sampling Human-in-the-loop

Return to Q327's situation and clause-by-clause explanation

Q328. A procurement agent extracts unit prices from supplier quotes and writes lines such as "Quote Q-118, page 3, dated 2 May, lists the unit price as $4.10." The ERP system that imports this output keeps the price but never keeps the quote reference, so buyers cannot check where a price came from. What is the root cause? A) The agent should maintain a separate lookup table from each price value to its quote so buyers can search it whenever they need to. B) The source reference exists only inside a sentence, and the consumer parses values, so the fix is emitting each price with separate source fields. C) The ERP import mapping is misconfigured, and adding a free-text notes column to carry each line through unchanged would solve it. D) The citations should be moved into numbered footnotes at the end of the output so they are clearly set apart from the price values.

Understand Q328 · the situation and each part of the question

The real situation behind this question

A manufacturer's procurement team negotiates hundreds of supplier quotes a month, and an agent extracts the prices so buyers can compare them in the ERP. When a supplier disputed an invoice, the buyer found the price in the ERP but had no way to trace which quote it came from. The agent's output had cited sources clearly, yet everything except the numbers disappeared on import.

Read the question in parts

Part 1 · keeps the price but never keeps the quote reference

The consumer retains only structured values and drops the text around them.

Part 2 · writes lines such as

The agent's output is prose, where the source is embedded in the wording rather than stored in fields.

What you are being asked to decide

Why does attribution vanish when output moves to another system? Recognising provenance as data that must be structured, not prose.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
ProvenanceThe quote id, page and date that belong with each extracted unit price.
Audit trailThe path a buyer follows from an ERP price back to the quote that stated it.

Study: Provenance — claim-source mappings through synthesis Provenance Audit trail

Understand Q328's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Downstream systems keep fields and discard prose. Attach source, location and date to each extracted value as structured data so attribution survives every hop.

Connect the design to the situation

Supplier prices imported into an ERP without their quote references.

Best answer: B. The ERP extracts numbers and discards surrounding text, so any attribution written as prose is lost. Emitting price, quote id, page and date as fields means provenance travels as data. A notes column keeps the sentence but leaves provenance unparseable and easy to drop again.

What each choice would do here

A · Why this does not meet the full requirement

The agent should maintain a separate lookup table from each price value to its quote so buyers can search it whenever they need to.

A separate lookup keyed on price values breaks when two quotes share a price.

A lookup keyed on the price value breaks as soon as two quotes list $4.10, and it drifts whenever the agent and the ERP disagree about rounding. Provenance ends up stored far from the value it describes.

B · Why this meets the requirement

The source reference exists only inside a sentence, and the consumer parses values, so the fix is emitting each price with separate source fields.

Correct: provenance must be fields on the value, not words around it.

Each price becomes a record with price, currency, quote id, page and date. The ERP imports every field, buyers can click from a price to its quote, and any later transformation keeps the link because it is part of the data.

C · Why this does not meet the full requirement

The ERP import mapping is misconfigured, and adding a free-text notes column to carry each line through unchanged would solve it.

A notes column preserves text but not a queryable link, so the next transformation can drop it.

A notes column might carry the sentence, but buyers would still have to read text to find the quote. Reports, filters and later exports treat notes as optional, so the link is lost again at the next step.

D · Why this does not meet the full requirement

The citations should be moved into numbered footnotes at the end of the output so they are clearly set apart from the price values.

Footnotes are still prose, which the ERP discards just like inline citations.

Numbered footnotes still sit in text the ERP never parses. Moving a citation from the middle of a sentence to the end of a document does not make it something the import can keep.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Price with provenance as fields

JSON
{
  "item": "M6 hex bolt",
  "unit_price": 4.1,
  "currency": "USD",
  "source": {
    "quote_id": "Q-118",
    "page": 3,
    "quote_date": "2025-05-02"
  }
}

The ERP imports every field, so the quote id stays attached to the price.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Why does attribution vanish when output moves to another system?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Provenance — claim-source mappings through synthesis

Study: Provenance — claim-source mappings through synthesis Provenance Audit trail

Return to Q328's situation and clause-by-clause explanation

Q329. Before deleting a feature flag, a developer needs every place in a large monorepo that reads NEW_CHECKOUT_ENABLED, including config files, tests and docs. The search will take dozens of grep calls and file reads. Which statement about handing this search to an exploration subagent is accurate? A) The subagent may read files that the main session's permission settings block, which makes the search more complete. B) The subagent always uses fewer total tokens than the same search in the main session, which is the main reason to delegate. C) The subagent gets a much larger context window than the main session, so it can hold the entire monorepo at once. D) The subagent's greps and file reads fill its own context, and only its concise list of findings enters the main session.

Understand Q329 · the situation and each part of the question

The real situation behind this question

A team is retiring a checkout feature flag that has spread through application code, configuration, test fixtures and documentation over two years. The developer doing the cleanup wants the main session focused on carefully removing each use. Searching the whole monorepo will produce hundreds of lines of grep output and many file reads, most of it irrelevant once the list of locations is known.

Read the question in parts

Part 1 · dozens of grep calls and file reads

The search generates a large amount of output that is mostly noise after the locations are found.

Part 2 · Which statement about handing this search

The question tests what delegation truly provides, not whether delegation is a good idea.

What you are being asked to decide

What do you actually gain by giving a broad search to a subagent? Naming context isolation as the benefit and rejecting myths about capacity, permissions and cost.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
SubagentA worker with its own context that performs the flag search and reports back.
Context isolationThe search noise stays in the subagent; the main session receives only the list.

Study: The Task tool — how subagents are actually born Study: Context management Subagent Context isolation Context window

Understand Q329's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Subagents give isolation, not extra capacity, extra permissions or guaranteed savings. Send noisy discovery to one and bring back only the concise findings the main work needs.

Connect the design to the situation

A monorepo-wide search for every reader of a feature flag before removing it.

Best answer: D. Delegation keeps the dozens of search results and file reads out of the main conversation, which stays clear for the deletion work. Subagents do not get a bigger window or broader permissions, and total token use is often higher, not lower.

What each choice would do here

A · Why this does not meet the full requirement

The subagent may read files that the main session's permission settings block, which makes the search more complete.

Subagents operate within the same permission rules, not above them.

Permission rules still apply to subagents, and a subagent cannot read files that settings deny. Any belief that delegation gets around restrictions would be a security problem, not a feature.

B · Why this does not meet the full requirement

The subagent always uses fewer total tokens than the same search in the main session, which is the main reason to delegate.

Delegation often costs more total tokens; the gain is a cleaner main context.

The subagent needs its own instructions and context, and the summary is extra output, so total tokens often go up. Delegating is worth it for the main session's clarity, not as a way to save tokens.

C · Why this does not meet the full requirement

The subagent gets a much larger context window than the main session, so it can hold the entire monorepo at once.

Subagents do not receive a larger window; they receive a separate one.

The subagent runs with a context window like any other session. If the monorepo is too large for one context, it is too large for the subagent too; its value comes from being separate, not bigger.

D · Why this meets the requirement

The subagent's greps and file reads fill its own context, and only its concise list of findings enters the main session.

Correct: the benefit is that the noisy search never enters the main context.

The subagent does the searching and reading in its own context and returns a list of files and lines that use the flag. The main session gains a short, accurate inventory and keeps its context for the removal work.

Check your understanding · explain the design in your own words

What do you actually gain by giving a broad search to a subagent?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The Task tool — how subagents are actually born Study: Context management

Study: The Task tool — how subagents are actually born Study: Context management Subagent Context isolation Context window

Return to Q329's situation and clause-by-clause explanation

Q330. Early in a long Claude Code session building a payments reconciliation job, the developer and the agent agreed that money amounts must always be integer minor units, never floating point. After several compactions, the agent writes total = float(row.amount). Which practice would most reliably have prevented this? A) Restate the rule once more in the chat halfway through the session so it is more recent when the next compaction happens. B) Compact more often, so that each summary is shorter and closer to the moment when the money-handling rule was agreed. C) Record the rule in the project's CLAUDE.md or a decisions file that is reloaded every session, not only in the chat history. D) Lower the temperature for the coding steps so the agent follows the patterns it established earlier in the session more closely.

Understand Q330 · the situation and each part of the question

The real situation behind this question

A fintech developer is using a long Claude Code session to build a job that reconciles bank payouts against internal ledgers. The first hour settled key decisions, including that money is always stored as integer cents to avoid rounding errors. Hours later, after several compactions, the agent introduced floating-point arithmetic. The bug was caught in review, but the developer wants to know how to keep such rules from eroding in future sessions.

Read the question in parts

Part 1 · After several compactions

The history has been summarised repeatedly, and each pass may drop precision from early decisions.

Part 2 · integer minor units, never floating point

The rule is precise and easy to blur into something vaguer during summarisation.

What you are being asked to decide

How do you keep an agreed rule intact through many compactions? Moving must-keep constraints out of compactable history into durable memory.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CompactionSummarising the conversation to free space, which can blur the exact money rule.
CLAUDE.mdProject memory loaded outside the conversation, so the rule survives compaction unchanged.

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests Compaction CLAUDE.md Scratchpad

Understand Q330's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Compaction preserves continuity but can blur exact rules. Keep constraints that must not drift in durable project memory or a decisions file that is reloaded, not only in the conversation.

Connect the design to the situation

A no-floats money rule lost after several compactions in a long coding session.

Best answer: C. Each compaction rewrites history into a shorter summary, and a precise rule can blur into something vague. Project memory such as CLAUDE.md is reloaded rather than summarised, so the rule stays exact. Restating it in chat only puts it back into history that will be compacted again.

What each choice would do here

A · Why this does not meet the full requirement

Restate the rule once more in the chat halfway through the session so it is more recent when the next compaction happens.

A restated rule still lives in history and faces the next compaction.

Restating the rule helps briefly, but it becomes part of the history the next compaction summarises. The developer would need to keep restating it, which is a manual workaround for a missing durable record.

B · Why this does not meet the full requirement

Compact more often, so that each summary is shorter and closer to the moment when the money-handling rule was agreed.

More frequent compaction adds more lossy rewrites, not fewer.

More frequent compaction means more rewrites of the same history, each one a chance to lose detail. The rule would pass through additional summaries and could blur sooner rather than later.

C · Why this meets the requirement

Record the rule in the project's CLAUDE.md or a decisions file that is reloaded every session, not only in the chat history.

Correct: durable memory keeps the rule exact regardless of what compaction does.

Writing the rule into CLAUDE.md or a decisions file means it is loaded as project memory, not rebuilt from summaries. After any number of compactions the agent still reads the exact rule and would reject float arithmetic.

D · Why this does not meet the full requirement

Lower the temperature for the coding steps so the agent follows the patterns it established earlier in the session more closely.

Temperature does not bring back a rule that compaction has blurred.

Lower temperature makes output more predictable but only reflects what is in context. If the summary now says handle money carefully, a deterministic agent still writes float arithmetic.

Check your understanding · explain the design in your own words

How do you keep an agreed rule intact through many compactions?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests

Study: When to summarize a long loop without losing tool results Study: Scratchpads & crash-recovery manifests Compaction CLAUDE.md Scratchpad

Return to Q330's situation and clause-by-clause explanation

Q331. An incident agent first reads 25 log files and metrics exports to reconstruct an outage, then writes the postmortem. By the writing phase its context is full of raw log lines, and the draft gets several timestamps and the order of events wrong. How should the workflow be structured? A) Move to a model tier with a larger context window so the raw logs and the postmortem draft fit together without crowding. B) End the investigation with a structured brief of timeline, evidence and file references, and write the postmortem from that brief. C) Interleave the phases so each paragraph of the postmortem is written right after the log file it relies on has been read. D) Have the writer reread all 25 files at the start of the writing phase so the timeline details are fresh in its context.

Understand Q331 · the situation and each part of the question

The real situation behind this question

An operations team uses an agent to draft postmortems after incidents. The agent reads logs, dashboards and alert histories, then writes the report. Recent drafts have had wrong timestamps and events out of order, which undermines trust in the process because on-call engineers must correct them. The raw material is correct; the problem appears at the point where the agent moves from investigating to writing.

Read the question in parts

Part 1 · its context is full of raw log lines

The writing phase is carrying everything from investigation, most of it irrelevant.

Part 2 · then writes the postmortem

There is a clear phase boundary where a handoff can be designed.

What you are being asked to decide

How should a two-phase task hand work from investigation to writing? Using a compact phase brief instead of carrying raw exploration output forward.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Context budgetThe limited space the writer has, which raw log lines were wasting.
ScratchpadThe investigation brief with the timeline and evidence, written before the writing phase starts.

Study: When to summarize a long loop without losing tool results Study: The budget mindset Context budget Scratchpad Context engineering

Understand Q331's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Carrying raw exploration output into the next phase buries the facts the next phase needs. Close each phase with a compact structured brief and begin the next one from it.

Connect the design to the situation

An outage postmortem drafted in a context crowded with raw log lines.

Best answer: B. The writer needs a clean timeline, not 25 files of raw lines. Closing the investigation with a structured brief and starting the writing phase from it gives the writer only what matters. Rereading every file or enlarging the window keeps the same noise in the writer's context.

What each choice would do here

A · Why this does not meet the full requirement

Move to a model tier with a larger context window so the raw logs and the postmortem draft fit together without crowding.

A bigger window holds more noise and costs more; the facts are still buried.

A larger window fits more raw lines, but the key timestamps are still scattered among thousands of irrelevant ones. Cost goes up and the writer faces the same search problem at a bigger scale.

B · Why this meets the requirement

End the investigation with a structured brief of timeline, evidence and file references, and write the postmortem from that brief.

Correct: the writer starts from distilled facts rather than raw output.

The investigation ends by writing a brief: a timeline with exact timestamps, key evidence and the files each fact came from. The writing phase starts from that brief, so the draft follows a clear sequence and each fact can be traced.

C · Why this does not meet the full requirement

Interleave the phases so each paragraph of the postmortem is written right after the log file it relies on has been read.

Interleaving prevents a clean overview, so the event order is still easy to get wrong.

Writing each paragraph right after reading its file gives no chance to see the whole timeline first. The postmortem needs the order of events across all sources, which interleaving makes harder to establish.

D · Why this does not meet the full requirement

Have the writer reread all 25 files at the start of the writing phase so the timeline details are fresh in its context.

Rereading all files puts the same raw noise straight back into context.

Rereading all 25 files floods the writer with raw lines again, exactly the state that caused the mistakes. Fresh noise is still noise, and the timeline must be extracted from it once more.

Check your understanding · explain the design in your own words

How should a two-phase task hand work from investigation to writing?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: When to summarize a long loop without losing tool results Study: The budget mindset

Study: When to summarize a long loop without losing tool results Study: The budget mindset Context budget Scratchpad Context engineering

Return to Q331's situation and clause-by-clause explanation

Q332. An agent has spent two hours investigating a legacy insurance-claims system across many tool calls. Which observation is the strongest sign of context degradation rather than some other problem? A) A reply stops in the middle of a sentence and the response shows a stop reason indicating the output token limit was reached. B) Early answers named exact files and classes such as ClaimAdjuster; recent answers on the same area say it "likely follows common patterns." C) Several consecutive tool calls fail with rate-limit responses, and the agent waits a while before trying the same calls again. D) The agent declines to print the database password stored in the configuration file and suggests using an environment variable instead.

Understand Q332 · the situation and each part of the question

The real situation behind this question

A consulting engineer is using an agent to map an old claims-processing system before a rewrite. The session has run for two hours and dozens of tool calls. The engineer has noticed that the agent's answers feel different lately and wants to know whether the cause is the context itself or something else, because the remedy differs: persisting findings and splitting work for degradation, versus configuration changes for the other problems.

Read the question in parts

Part 1 · across many tool calls

The session has accumulated a lot of output, the condition in which degradation appears.

Part 2 · the strongest sign of context degradation

The task is to tell degradation apart from other failures that look similar at first.

What you are being asked to decide

Which symptom points to context degradation specifically? Diagnosing degradation from its symptom and separating it from look-alike failures.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Context windowThe space filled by two hours of tool output, pushing earlier specifics out of focus.
ScratchpadWhere the agent should have written findings like class names so they survive a long session.

Study: Context management Study: Scratchpads & crash-recovery manifests Context window Context engineering Scratchpad

Understand Q332's answer · full reasoning and worked examples
Claude synthesized · the point, straight

The signature of context degradation is specifics giving way to generalities about things the agent once knew precisely. Other failures such as output caps, deliberate refusals and rate limits have different signs and different fixes.

Connect the design to the situation

A long legacy-code investigation where answers lose their specific class names.

Best answer: B. Degradation shows as a shift from specific, cited details to vague generalisations about things the agent knew precisely earlier. A cut-off reply is an output limit, a refusal to print secrets is intended behaviour, and rate-limit errors are an external service problem.

What each choice would do here

A · Why this does not meet the full requirement

A reply stops in the middle of a sentence and the response shows a stop reason indicating the output token limit was reached.

A max-tokens stop is an output length limit, not a loss of earlier knowledge.

The reply was cut off because it hit the output token cap, visible in the stop reason. Raising the limit or continuing the reply fixes it; nothing has been lost from earlier in the session.

B · Why this meets the requirement

Early answers named exact files and classes such as ClaimAdjuster; recent answers on the same area say it "likely follows common patterns."

Correct: the drift from exact names to generic phrases is the classic degradation symptom.

The agent once cited `ClaimAdjuster` and exact file paths and now gives vague descriptions of the same code. That drift is what happens when accumulated output crowds out earlier findings, and it calls for persisted notes and fresh or delegated contexts.

C · Why this does not meet the full requirement

Several consecutive tool calls fail with rate-limit responses, and the agent waits a while before trying the same calls again.

Rate limits come from the service, not from the state of the agent's context.

Rate-limit responses come from the service being called and resolve with waiting and backoff. They say nothing about whether the agent still remembers what it found earlier.

D · Why this does not meet the full requirement

The agent declines to print the database password stored in the configuration file and suggests using an environment variable instead.

Refusing to reveal a secret is intended safety behaviour, not degradation.

Declining to print a password and suggesting an environment variable is deliberate, secure behaviour. It is unrelated to how full the context is and would happen at the start of a session too.

Check your understanding · explain the design in your own words

Which symptom points to context degradation specifically?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Context management Study: Scratchpads & crash-recovery manifests

Study: Context management Study: Scratchpads & crash-recovery manifests Context window Context engineering Scratchpad

Return to Q332's situation and clause-by-clause explanation

Q333. A due-diligence agent assembles an acquisition report from three sources: court-records search, financial filings and press coverage. On one run the court-records service was down, and the report went to the client looking complete. Which output design is right? A) Tag each section with which sources contributed and flag that litigation coverage is missing and related conclusions are weaker. B) Have the agent fill the litigation section from its general knowledge of the target company, labelled as a model estimate. C) Hold the report entirely until all three sources are available again, returning only an error message to the client until then. D) Put a standard notice on every report saying that sources may be incomplete, so clients always read results with some caution.

Understand Q333 · the situation and each part of the question

The real situation behind this question

An advisory firm produces acquisition due-diligence reports for private-equity clients on tight deal timelines. During one run the court-records service was unavailable, and the report went out without any litigation findings. The client read it as saying there was no litigation, which is a serious mistake to make about an acquisition target. The firm wants reports that remain useful when a source fails but never imply coverage they do not have.

Read the question in parts

Part 1 · went to the client looking complete

The failure was invisible, which is what made the output misleading.

Part 2 · court-records search, financial filings and press coverage

There are three sources, so a report can show exactly which one is missing.

What you are being asked to decide

How should a report show that one of its sources was unavailable? Choosing specific coverage annotations over blocking, invented filler or generic disclaimers.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Graceful degradationDelivering the financial and press findings while clearly marking the missing litigation part.
ProvenanceRecording which sources supported each section so gaps are visible.

Study: Resilience patterns Study: Provenance — claim-source mappings through synthesis Graceful degradation Provenance

Understand Q333's answer · full reasoning and worked examples
Claude synthesized · the point, straight

When a source fails, deliver what is supported and state precisely what is missing. Coverage annotations per section keep the output useful and honest, unlike blocking, invented filler or generic disclaimers.

Connect the design to the situation

An acquisition report delivered without court records and without saying so.

Best answer: A. Coverage annotations tell the client exactly which part lacks evidence, so they can use the rest and wait for or request the missing part. A blanket notice on every report says nothing specific, and filling the gap from the model's own knowledge presents unretrieved content as findings.

What each choice would do here

A · Why this meets the requirement

Tag each section with which sources contributed and flag that litigation coverage is missing and related conclusions are weaker.

Correct: section-level coverage shows exactly what is supported and what is missing.

The report shows that the financial and press sections are fully sourced, the litigation section was not searched, and any risk conclusion depending on litigation is provisional. The client knows exactly what to follow up on.

B · Why this does not meet the full requirement

Have the agent fill the litigation section from its general knowledge of the target company, labelled as a model estimate.

Model knowledge is not retrieved evidence, and a label does not make it a finding.

General knowledge of the target is not a court-records search and may be outdated or wrong. A label helps, but it still places unverified material where the client expects retrieved evidence about lawsuits.

C · Why this does not meet the full requirement

Hold the report entirely until all three sources are available again, returning only an error message to the client until then.

Blocking everything withholds two thirds of good evidence the client could use now.

Holding the report protects against misreading but costs the client time on a live deal. The financial and press findings are sound, and the client could act on them with the gap clearly marked.

D · Why this does not meet the full requirement

Put a standard notice on every report saying that sources may be incomplete, so clients always read results with some caution.

A generic notice on every report tells the client nothing about this specific gap.

A standard notice appears whether or not anything failed, so clients learn to ignore it. It does not say that litigation specifically was not searched, which is the fact the client needed.

Check your understanding · explain the design in your own words

How should a report show that one of its sources was unavailable?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Resilience patterns Study: Provenance — claim-source mappings through synthesis

Study: Resilience patterns Study: Provenance — claim-source mappings through synthesis Graceful degradation Provenance

Return to Q333's situation and clause-by-clause explanation

Q334. A news-briefing pipeline has three collector agents that each return articles with URLs. To save tokens, a condenser step turns each collector's output into short bullet highlights before an editor agent writes the daily brief. The brief never contains links, even though every collector output did. Which change fixes this at the right point? A) Have the condenser output each highlight as a record paired with its source URLs, and have the editor keep those pairs in the brief. B) Add an instruction to the editor's system prompt saying that every statement in the brief must include a link to its source. C) After the brief is written, run a step that searches the web for each sentence and attaches the best-matching article link to it. D) Give the editor a larger context window so it can take in all collector outputs directly and cite from them in full.

Understand Q334 · the situation and each part of the question

The real situation behind this question

A media-monitoring company sends clients a daily brief built from three collector agents covering regulators, competitors and trade press. Clients complained they cannot verify claims because the brief has no links. The engineers found that every collector returns URLs, and the editor receives all the condenser's output. Somewhere between the two the links disappear, and the team needs to fix it without losing the token savings the condenser provides.

Read the question in parts

Part 1 · a condenser step turns each collector's output into short bullet highlights

The summarising step is the point where source links can be dropped.

Part 2 · even though every collector output did

Links exist upstream, so they are lost in the pipeline, not missing at collection.

What you are being asked to decide

Where in the pipeline are the links lost, and where should the fix go? Locating provenance loss at the summarisation step and fixing it there.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
ProvenanceThe link from each highlight back to the article URL it came from.
Context engineeringDeciding what the condenser passes forward so the editor has what it needs.

Study: Provenance — claim-source mappings through synthesis Study: When to summarize a long loop without losing tool results Provenance Context engineering

Understand Q334's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Provenance disappears at whatever step summarises without keeping source links. Require claim-to-source records at that step and carry them through synthesis, instead of asking later stages to recreate links they never received.

Connect the design to the situation

A condenser step turns sourced articles into bullets with no URLs before the editor writes.

Best answer: A. The links disappear in the condenser, which reduces sourced articles to plain bullets, so the editor has nothing to cite. Making the condenser emit highlight-to-source records fixes the loss where it happens. Telling the editor to cite asks for links it never receives, and searching afterwards attaches guessed sources.

What each choice would do here

A · Why this meets the requirement

Have the condenser output each highlight as a record paired with its source URLs, and have the editor keep those pairs in the brief.

Correct: the mapping is preserved at the step that was dropping it.

The condenser still saves tokens by shortening article text, but each highlight carries its URLs in a field. The editor receives claims already paired with sources, and the brief can cite every statement.

B · Why this does not meet the full requirement

Add an instruction to the editor's system prompt saying that every statement in the brief must include a link to its source.

The editor cannot cite links the condenser never passed on.

The editor is being asked to link sources it has never seen. It may omit links or, worse, invent plausible URLs, because the condensed bullets contain no source information.

C · Why this does not meet the full requirement

After the brief is written, run a step that searches the web for each sentence and attaches the best-matching article link to it.

Searching afterwards finds plausible articles, not the ones actually used.

Web searches after the fact attach whichever article best matches a sentence, which may not be the one the collector used. Clients get links that look authoritative but do not reliably support the claims.

D · Why this does not meet the full requirement

Give the editor a larger context window so it can take in all collector outputs directly and cite from them in full.

Bypassing the condenser gives up the token savings the pipeline was built for.

Sending all collector output straight to the editor might preserve links, but it removes the condenser and its token savings, and raw articles crowd the editor's context. It fixes the symptom by abandoning the design.

Check your understanding · explain the design in your own words

Where in the pipeline are the links lost, and where should the fix go?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Provenance — claim-source mappings through synthesis Study: When to summarize a long loop without losing tool results

Study: Provenance — claim-source mappings through synthesis Study: When to summarize a long loop without losing tool results Provenance Context engineering

Return to Q334's situation and clause-by-clause explanation

Q335. A contract-review pipeline sends whole 120-page service agreements in one request. Clauses near the start and end are extracted reliably, but indemnification terms, which usually sit around pages 55–70, are missed in most contracts. Token counts confirm nothing is truncated. Which change targets the actual cause? A) Increase max_tokens on the request so the model has more room to write out the indemnification clauses it finds. B) Repeat the instruction to extract indemnification terms twice in the system prompt so the model gives it more importance. C) Split each agreement by section and run a focused extraction pass over the middle sections, placing the question after that text. D) Move to a model with a larger context window so each 120-page agreement uses a smaller fraction of the space available.

Understand Q335 · the situation and each part of the question

The real situation behind this question

A legal-operations team uses an extraction pipeline to review supplier agreements for risky terms before renewal. Their audit found that indemnification clauses, among the most important terms, are missed in most contracts, while definitions at the front and signatures at the back are captured reliably. Engineers confirmed that each contract fits in the request with room to spare, so the failure is not about text being cut off.

Read the question in parts

Part 1 · Token counts confirm nothing is truncated

The full contract is in context, which rules out truncation as the cause.

Part 2 · around pages 55–70

The missed content sits in the middle of the input, where position effects are strongest.

What you are being asked to decide

Why is the middle of a long contract missed when nothing is cut off? Recognising position effects in long inputs and choosing a structural fix.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Long-context placementWhere text sits in a long prompt, which affects how much attention it gets.
Context windowAlready large enough here, which is why a bigger one does not help.

Study: Lost in the middle — position effects Long-context placement Context window

Understand Q335's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Content in the middle of long inputs gets less attention even when it fits. Restructure with section-focused passes and place key material and questions at the edges rather than buying a bigger window.

Connect the design to the situation

Indemnification terms deep in 120-page agreements are missed despite full inclusion.

Best answer: C. Reliable extraction at the start and end with misses in the middle, and no truncation, is the lost-in-the-middle position effect. Focused passes on the middle sections put that text where attention is strong. A larger window changes nothing, since everything already fits.

What each choice would do here

A · Why this does not meet the full requirement

Increase max_tokens on the request so the model has more room to write out the indemnification clauses it finds.

max_tokens limits the reply length, not how well the input is read.

Raising the output limit gives the model more space to answer but does nothing to how it reads the input. Clauses it overlooked will still be overlooked, just with a longer reply.

B · Why this does not meet the full requirement

Repeat the instruction to extract indemnification terms twice in the system prompt so the model gives it more importance.

Repeating the instruction does not change the attention given to middle pages.

Repeating an instruction may raise its general emphasis, but it does not change where the clauses sit in a 120-page input. The model still attends weakly to the middle and misses them as before.

C · Why this meets the requirement

Split each agreement by section and run a focused extraction pass over the middle sections, placing the question after that text.

Correct: focused passes put the middle content where attention is strong.

Splitting by section lets a pass read the middle sections alone, with the extraction request placed after that text. The indemnification clauses are now at the edge of a short input, where they are reliably found.

D · Why this does not meet the full requirement

Move to a model with a larger context window so each 120-page agreement uses a smaller fraction of the space available.

Nothing is truncated, so more room does not change where the clauses sit.

The contract already fits, so a larger window leaves its layout unchanged. Indemnification text still sits in the middle of a long input and receives the same weaker attention.

Check your understanding · explain the design in your own words

Why is the middle of a long contract missed when nothing is cut off?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Lost in the middle — position effects

Study: Lost in the middle — position effects Long-context placement Context window

Return to Q335's situation and clause-by-clause explanation

Q336. A support platform auto-applies ticket categories when model confidence exceeds 0.85; a single labelled validation run showed 96% accuracy. The company launches new product lines every quarter, and operations wants to catch new mistakes in the auto-applied categories early. Which monitoring plan does that best? A) Rerun the original validation set every month and raise an alert if its measured accuracy ever drops below 96%. B) Count how often support agents manually recategorise tickets and investigate whenever that number rises noticeably. C) Each week, have people label a random sample of auto-applied categories drawn from every product line, and track error rates per line. D) Track the average confidence of auto-applied categories each week, and alert when the average falls below its level at launch.

Understand Q336 · the situation and each part of the question

The real situation behind this question

A software company routes support tickets by category, and an automatic classifier handles most of them. Each quarter brings new products with new vocabulary, and misrouted tickets wait longer and anger customers. Accuracy was measured once at launch and looked excellent. Operations leaders want a way to learn early when the classifier starts making new kinds of mistakes, rather than finding out months later from support metrics.

Read the question in parts

Part 1 · a single labelled validation run

The only evidence is a one-time measurement that predates the new product lines.

Part 2 · launches new product lines every quarter

The input population keeps changing, so any static check goes stale.

What you are being asked to decide

How do you keep checking automated output as the inputs change? Choosing live stratified sampling over self-reported signals, stale test sets and complaints.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
Stratified samplingDrawing weekly review samples from every product line, including the newest.
CalibrationChecking that a confidence above 0.85 still matches the accuracy people actually measure.

Study: Confidence calibration & stratified sampling (human-review routing) Stratified sampling Calibration Human-in-the-loop

Understand Q336's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Assurance about automated decisions has to come from humans checking fresh samples of those decisions. Sample auto-approved output by segment every period and track error rates, instead of trusting confidence averages, stale test sets or complaint counts.

Connect the design to the situation

Auto-applied ticket categories at a company that adds product lines each quarter.

Best answer: C. Only fresh human labels on the population actually being auto-applied measure its real accuracy, and stratifying by product line lets a new line's errors appear quickly. Average confidence is self-reported and can stay high while accuracy falls; the old validation set never contains the new products.

What each choice would do here

A · Why this does not meet the full requirement

Rerun the original validation set every month and raise an alert if its measured accuracy ever drops below 96%.

The old set never includes new product lines, so it cannot reveal their errors.

The original validation set holds tickets from launch-era products only. It can score 96% forever while tickets about new products are misrouted, because none of those tickets are in it.

B · Why this does not meet the full requirement

Count how often support agents manually recategorise tickets and investigate whenever that number rises noticeably.

Manual fixes only capture mistakes agents notice and bother to correct.

Recategorisation counts depend on agents noticing and fixing mistakes, which many do not bother to do under time pressure. They are a lagging and incomplete signal, not a measurement of accuracy.

C · Why this meets the requirement

Each week, have people label a random sample of auto-applied categories drawn from every product line, and track error rates per line.

Correct: fresh labels on live auto-applied output, by segment, show real error rates.

Weekly labelled samples from each product line show an error rate for every segment. When a new line launches, its errors appear in its own stratum within a week or two, long before overall metrics shift.

D · Why this does not meet the full requirement

Track the average confidence of auto-applied categories each week, and alert when the average falls below its level at launch.

Average confidence is self-reported and can stay high while the model is confidently wrong.

Average confidence reflects what the model believes about itself. On a new product it can label tickets confidently and wrongly, so the average stays high while misrouted tickets pile up unnoticed.

Check your understanding · explain the design in your own words

How do you keep checking automated output as the inputs change?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Confidence calibration & stratified sampling (human-review routing)

Study: Confidence calibration & stratified sampling (human-review routing) Stratified sampling Calibration Human-in-the-loop

Return to Q336's situation and clause-by-clause explanation

Q337. A fleet-maintenance agent checks 300 vehicles in one run. Each get_vehicle_status call returns about 60 sensor fields, but the task only uses battery_pct, odometer_km and fault_codes. Accuracy drops noticeably for vehicles late in the run. Where should the fix go? A) In the session settings, compacting the conversation after every 50 vehicles so the accumulated tool output is summarised. B) In the request parameters, raising max_tokens so the model has more space to reason about each vehicle's full status. C) In the system prompt, with a line telling the model to ignore every sensor field except the three that matter for maintenance. D) In the tool-result handling, reducing each response to the three needed fields before it is added to the conversation.

Understand Q337 · the situation and each part of the question

The real situation behind this question

A logistics company runs a nightly agent that reviews every delivery van for maintenance needs. The telemetry service returns a large status payload for each vehicle, and the agent processes hundreds in one run. Maintenance planners noticed that recommendations for vans near the end of the list are worse than for vans near the start. The team suspects context growth and wants the fix in the right place.

Read the question in parts

Part 1 · returns about 60 sensor fields

Each tool result is far larger than the task requires.

Part 2 · only uses `battery_pct`, `odometer_km` and `fault_codes`

The needed data is known in advance, so trimming is safe and simple.

What you are being asked to decide

Where should unneeded tool output be removed? Placing the trim at the tool-result boundary rather than relying on instructions or later compaction.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
tool_resultThe vehicle status payload appended to the conversation after each call.
Token optimizationKeeping only the three fields the task uses so context stays lean.

Study: The budget mindset Study: When to summarize a long loop without losing tool results Token optimization tool_result Hook

Understand Q337's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Unused tool output crowds context over long runs. Trim results to the needed fields before they are appended, rather than instructing the model to ignore them or cleaning up afterwards.

Connect the design to the situation

Sixty-field vehicle status payloads accumulating over a 300-vehicle run.

Best answer: D. The unused fields fill the context across hundreds of calls and crowd out earlier results. Reducing each result to the needed fields before it is appended stops the bloat at its source. Telling the model to ignore fields still leaves every token in context, and compaction is later and lossy.

What each choice would do here

A · Why this does not meet the full requirement

In the session settings, compacting the conversation after every 50 vehicles so the accumulated tool output is summarised.

Periodic compaction reacts after the bloat builds up and loses detail.

Compacting every 50 vehicles clears some space, but only after each batch has already crowded the context. Summaries of earlier vehicles lose details, and the run pays for the bloat before cleaning it up.

B · Why this does not meet the full requirement

In the request parameters, raising max_tokens so the model has more space to reason about each vehicle's full status.

max_tokens limits the reply; it does nothing about input growth.

A larger output limit lets the model write more for each vehicle but does nothing about input growth. The crowding comes from accumulated tool results, which max_tokens does not affect.

C · Why this does not meet the full requirement

In the system prompt, with a line telling the model to ignore every sensor field except the three that matter for maintenance.

An instruction to ignore fields still leaves all 60 fields occupying context.

The model may ignore the fields when reasoning, but each response still adds all 60 fields to the context. After a few hundred vehicles the context is equally full, and the late vehicles degrade just as before.

D · Why this meets the requirement

In the tool-result handling, reducing each response to the three needed fields before it is added to the conversation.

Correct: the unused fields never enter the conversation at all.

A wrapper or post-tool hook keeps only the three needed fields from each response. Each vehicle adds a few lines instead of a large payload, so the context stays small across all 300 vehicles and earlier results remain sharp.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Trim before appending

PYTHON
KEEP = ('battery_pct', 'odometer_km', 'fault_codes')

def trim_status(raw: dict) -> dict:
    return {k: raw.get(k) for k in KEEP}

# in the loop, append trim_status(result) as the tool_result content, not result
🏭 Production shape: the tool returns the trimmed result

In production the trim happens inside the tool function, so nothing else can append the raw payload. The @beta_tool decorator builds the definition, and the runner appends only what the function returns.

PYTHON
import json
import os
from typing import Annotated

import anthropic
from anthropic import beta_tool
from pydantic import Field

from app.fleet import telematics  # your own client for the vehicle API

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
MODEL = os.environ.get("CLAUDE_MODEL", "claude-sonnet-5")
KEEP = ("battery_pct", "odometer_km", "fault_codes")


@beta_tool(
    name="get_vehicle_status",
    description="Read one vehicle's battery level, odometer and active fault codes.",
)
def get_vehicle_status(vehicle_id: Annotated[str, Field(description="The fleet vehicle id, e.g. EV-0412.")]) -> str:
    raw = telematics.status(vehicle_id)  # about sixty fields
    return json.dumps({k: raw.get(k) for k in KEEP})


runner = client.beta.messages.tool_runner(
    model=MODEL, max_tokens=2048, tools=[get_vehicle_status],
    messages=[{"role": "user", "content": "Which vehicles need service today?"}],
    max_iterations=50,
)
final = runner.until_done()

The tool lives in app/tools/fleet.py; the runner call in app/agent.py.

Each vehicle contributes three fields to context instead of sixty.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

Where should unneeded tool output be removed?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: The budget mindset Study: When to summarize a long loop without losing tool results

Study: The budget mindset Study: When to summarize a long loop without losing tool results Token optimization tool_result Hook

Return to Q337's situation and clause-by-clause explanation

Q338. A mortgage-application extractor gets applicant_name and loan_amount right almost every time, but employment_start_date and self-employed income are often wrong. Underwriters can review roughly 15% of extracted fields. Which routing design spends that capacity where the errors are? A) Emit a confidence score for each field, set thresholds per field against a labelled set, and review only fields below them. B) Send every application that includes self-employed income to review in full, so underwriters see all fields with context. C) At the end of each application, ask the model which applications it feels unsure about overall and send those to underwriters. D) Pick a random 15% of all applications each day and have underwriters check every field in each application picked.

Understand Q338 · the situation and each part of the question

The real situation behind this question

A lender's underwriting team is overloaded, and an extractor now pre-fills application data from pay stubs, tax returns and bank statements. Some fields are nearly perfect, while employment dates and self-employed income are often wrong, and those are the fields that drive approval decisions. With only enough reviewer time for about 15% of fields, the lender needs routing that puts human attention exactly where errors are likely.

Read the question in parts

Part 1 · roughly 15% of extracted fields

Review capacity is limited and counted in fields, so routing should work at field level.

Part 2 · `employment_start_date` and self-employed income are often wrong

Error rates differ by field, which uniform or whole-application routing ignores.

What you are being asked to decide

How do you target scarce reviewers at the fields most likely to be wrong? Routing by calibrated per-field confidence instead of self-assessment or whole-document routing.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CalibrationSetting each field’s threshold so its confidence score matches measured accuracy on labelled applications.
Human-in-the-loopUnderwriters reviewing only the fields that fall below their calibrated threshold.

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design Calibration Human-in-the-loop Stratified sampling

Understand Q338's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Scarce review capacity belongs on the fields most likely to be wrong. Emit per-field confidence, calibrate each field's threshold against labelled data and route only fields below it, never the model's own uncalibrated sense of doubt.

Connect the design to the situation

A mortgage extractor whose employment-date and income fields are error-prone.

Best answer: A. Per-field scores with thresholds calibrated on labelled applications send reviewers to the specific fields likely to be wrong. The model's own overall sense of unease is uncalibrated self-assessment, and whole-application or random routing spends most reviewer time on fields that are almost always right.

What each choice would do here

A · Why this meets the requirement

Emit a confidence score for each field, set thresholds per field against a labelled set, and review only fields below them.

Correct: calibrated per-field thresholds send reviewers to the fields likely to be wrong.

Each field carries a score, and thresholds are set per field so that a flagged employment date really is likely to be wrong. Underwriters see only those fields, with the source excerpt, and the 15% capacity covers the fields that matter.

B · Why this does not meet the full requirement

Send every application that includes self-employed income to review in full, so underwriters see all fields with context.

Whole-application review spends effort on name and amount fields that are almost always right.

Every self-employed application goes to full review, including name and loan amount fields that are almost never wrong. Capacity runs out quickly and employment-date errors in other applications are never checked.

C · Why this does not meet the full requirement

At the end of each application, ask the model which applications it feels unsure about overall and send those to underwriters.

The model's overall unease is uncalibrated and ignores which fields are risky.

The model's sense of overall unease is not calibrated against outcomes and may miss confidently wrong income figures. It also sends whole applications, so reviewers spend time on reliable fields like applicant name.

D · Why this does not meet the full requirement

Pick a random 15% of all applications each day and have underwriters check every field in each application picked.

Random sampling measures quality but does not target review at weak fields.

A random 15% of applications gives an unbiased quality measure, which is useful for monitoring. As routing, though, it spreads reviewer effort evenly, so most error-prone fields in the other 85% are never reviewed.

Walk through complete payloads or code · every step explained

Illustrative examples, not recorded live calls. Check each block's direction and endpoint: API requests, API responses, MCP messages, and configuration files are different contracts.

Per-field thresholds from a labelled set

JSON
{
  "applicant_name": {
    "threshold": 0.6,
    "measured_accuracy_above": 0.995
  },
  "loan_amount": {
    "threshold": 0.7,
    "measured_accuracy_above": 0.99
  },
  "employment_start_date": {
    "threshold": 0.92,
    "measured_accuracy_above": 0.97
  },
  "self_employed_income": {
    "threshold": 0.95,
    "measured_accuracy_above": 0.96
  }
}

Error-prone fields get stricter thresholds, so more of them are routed to underwriters.

Which omitted settings use defaults or leave features unconfigured?

Check your understanding · explain the design in your own words

How do you target scarce reviewers at the fields most likely to be wrong?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design

Study: Confidence calibration & stratified sampling (human-review routing) Study: Escalation design Calibration Human-in-the-loop Stratified sampling

Return to Q338's situation and clause-by-clause explanation

Q339. A warehouse agent reads delivery paperwork. Printed domestic packing slips are 70% of volume at 98.8% accuracy, customs forms are 20% at 96.9%, and handwritten slips are 10% at 61%. Overall accuracy is about 94.6%. The plan is to auto-post every extraction with confidence of at least 0.9. Which fact makes the plan unsafe? A) Inventory reconciliation will not catch errors quickly enough, so validation checks after posting must be added first. B) A single global threshold would let handwritten slips through automatically, even though roughly two in five of them are wrong. C) The 0.9 threshold is too low for all three paperwork types, so it should simply be raised to 0.95 for every type. D) Handwritten slips are only 10% of volume, so their errors are too small a share of the total to matter for the plan.

Understand Q339 · the situation and each part of the question

The real situation behind this question

A distribution centre handles thousands of deliveries a day, and an agent extracts item counts and SKUs from paperwork so stock can be posted without manual entry. Operations wants to automate high-confidence extractions to free clerks. The headline accuracy looks strong, but the paperwork is not uniform: some suppliers still send handwritten slips. Wrong postings cause stock-outs and costly recounts, so the team must know where automation is safe.

Read the question in parts

Part 1 · handwritten slips are 10% at 61%

One segment is far less accurate than the rest, even though it is small.

Part 2 · every extraction with confidence of at least 0.9

The plan applies one global threshold across all paperwork types.

What you are being asked to decide

Why is automating on overall accuracy unsafe here? Seeing how aggregate accuracy hides a weak segment and routing by segment.

Keep all the stated requirements in view. An option may be useful elsewhere while addressing only part of this situation.

Words and related concepts · open glossary explanations
TermMeaning here
CalibrationSetting thresholds per paperwork type from measured accuracy, not one global value.
Human-in-the-loopKeeping people reviewing handwritten slips until their accuracy reaches an acceptable level.

Study: Confidence calibration & stratified sampling (human-review routing) Calibration Stratified sampling Human-in-the-loop

Understand Q339's answer · full reasoning and worked examples
Claude synthesized · the point, straight

Overall accuracy is an average that high-volume segments dominate. Break accuracy down by input type and route weak segments to human review before automating anything with a single global threshold.

Connect the design to the situation

Handwritten slips at 61% hidden inside a 94.6% overall accuracy figure.

Best answer: B. The overall figure is carried by high-volume printed slips and hides that handwritten slips run at 61%. Segmenting by paperwork type and keeping handwritten slips under human review until they improve is what makes automation safe. A global threshold raise still judges all types by one uncalibrated bar.

What each choice would do here

A · Why this does not meet the full requirement

Inventory reconciliation will not catch errors quickly enough, so validation checks after posting must be added first.

Catching errors after posting is a detective control where a preventive one is needed.

Post-posting checks catch some errors after inventory is already wrong, causing stock-outs and recounts. Preventing handwritten errors from being auto-posted is cheaper than detecting them later.

B · Why this meets the requirement

A single global threshold would let handwritten slips through automatically, even though roughly two in five of them are wrong.

Correct: the global plan automates a segment that is wrong nearly 40% of the time.

Handwritten slips would be posted automatically whenever they scored 0.9, and at 61% accuracy about two in five would put wrong counts into inventory. The segment needs mandatory review until it improves, while printed slips can be automated.

C · Why this does not meet the full requirement

The 0.9 threshold is too low for all three paperwork types, so it should simply be raised to 0.95 for every type.

A higher global bar still ignores that each type needs its own calibrated threshold.

Raising the bar to 0.95 everywhere sends many accurate printed slips to clerks and still does not measure what a score means for handwriting. Handwritten slips can score 0.95 and be wrong if confidence was never calibrated for them.

D · Why this does not meet the full requirement

Handwritten slips are only 10% of volume, so their errors are too small a share of the total to matter for the plan.

Ten percent of volume at 61% means many wrong postings every day.

Ten percent of thousands of daily deliveries is hundreds of slips, and 39% of those wrong means many incorrect postings every day. Share of volume does not make an error rate acceptable.

Check your understanding · explain the design in your own words

Why is automating on overall accuracy unsafe here?

Describe the situation, identify the controlling requirement, and explain what the chosen design does before and after the critical step. Then choose one alternative and explain which part of this situation it fails to address. Use the example rather than memorizing the answer letter.

Study: Confidence calibration & stratified sampling (human-review routing)

Study: Confidence calibration & stratified sampling (human-review routing) Calibration Stratified sampling Human-in-the-loop

Return to Q339's situation and clause-by-clause explanation