Chapter 20 Agent Design Patterns

Building effective agents requires more than a powerful model and a set of tools. The architecture—how the LLM is orchestrated, how tasks are decomposed, and how control flows between components—determines whether an agent is reliable, debuggable, and cost-effective. This chapter presents the canonical design patterns that have emerged from production deployments at Anthropic, OpenAI, Google, and the open-source community.

Workflow Patterns

These patterns—adapted from Anthropic’s taxonomy of agentic building blocks [9]—use LLMs within a predefined control flow. The system (not the model) decides the execution order.

Prompt Chaining

The simplest pattern: break a complex task into a fixed sequence of LLM calls, piping the result of one call as context into the next. Validation gates between steps catch errors early before they propagate downstream.

When to use: Tasks that are naturally sequential—content generation, data transformation, multi-stage analysis.

Key advantage: Each step can use a different prompt, model, or temperature. Intermediate results are inspectable and debuggable.

Routing

A classifier (LLM or traditional) examines the input and dispatches to a specialized handler.

When to use: Distinct task types with different optimal prompts, tools, or models. Customer support triage, multi-modal input handling.

Parallelization

Multiple LLM calls run concurrently, with a programmatic layer combining their outputs. Two sub-patterns emerge:

• Sectioning (fan-out): Partition the input into disjoint chunks and process each independently—e.g., run security, performance, and style checks on a codebase simultaneously.
• Voting (redundancy): Issue the same prompt $N$ times with different seeds or temperatures, then select the best result via majority vote [368], reward-model scoring, or LLM-as-judge.

Orchestrator-Workers

Here the LLM itself decides how to split the work. An orchestrator model analyzes the task, produces a plan of subtasks, dispatches each subtask to a worker LLM (potentially with different prompts or tools), and finally merges their outputs into a coherent result. The key difference from parallelization is that the decomposition logic is model-generated, not hard-coded.

When to use: Open-ended problems where the number and nature of subtasks cannot be enumerated at design time—e.g., “refactor this codebase” requires first understanding the dependency graph before deciding which files to modify.

Evaluator-Optimizer

A two-model feedback loop [246]: a generator produces candidate outputs while a separate evaluator scores them against explicit criteria. If the score falls below a threshold, the evaluator’s critique is appended to the generator’s context and the cycle repeats until the quality bar is met or a retry budget is exhausted.

When to use: Tasks with clear quality criteria—code that must pass tests, translations that must preserve meaning, writing that must match a style guide.

Autonomous Agent Patterns

These patterns give the LLM control over the execution flow itself.

ReAct (Reason + Act)

The foundational agent pattern [409]. The LLM alternates between thinking (internal reasoning), acting (tool calls), and observing (processing results) in a loop until it produces a final answer.

Planning Agents

The agent generates an explicit plan before executing, and can revise the plan mid-execution [364].

Reflection and Self-Critique

The agent pauses to evaluate its own trajectory and correct course:

1. Output validation: “Is this correct? Did I miss anything?”
2. Trajectory review: Review last $k$ steps, identify mistakes or inefficiencies.
3. Strategy revision: Reconsider the overall approach (“Am I solving the right problem?”).

Tool-Use Patterns

How an agent invokes tools significantly affects its reliability, latency, and cost. Five canonical patterns have emerged [315]:

The simplest pattern: the model issues one tool call, receives the result, and produces a final answer. Sufficient for factual lookups, unit conversions, or single API queries. The harness makes exactly two LLM calls (one to decide on the tool, one to synthesize the result).

Modern APIs (OpenAI, Anthropic) allow the model to request multiple tool calls in a single response. The harness executes them concurrently and returns all results together. This dramatically reduces latency for tasks requiring independent information from multiple sources—e.g., fetching stock price, weather, and calendar simultaneously. The key constraint: the tools must be independent (no tool’s output is needed as input to another).

Each tool’s output feeds into the next tool’s input, forming a data pipeline. The model decides the next tool based on the previous result. Common in research workflows: search$\to$fetch_page$\to$extract_data$\to$analyze. The harness must track the growing context and may need to summarize intermediate results to stay within budget.

A tool call invokes an entirely separate agent—with its own prompt, tools, and context. The parent agent treats the sub-agent as a black-box function. This enables specialization: a research agent delegates code execution to a coding agent, which has access to a sandbox and test runner. The Swarm pattern [276] generalizes this via handoffs between specialized agents.

The harness tries tools in priority order: if the preferred tool fails (timeout, rate limit, API error), it automatically falls back to an alternative. The model need not be aware of the fallback logic—the harness handles it transparently. Example: primary search API $\to$ backup search $\to$ cached results $\to$ inform model that search is unavailable.

Design Principles

The following principles, distilled from Anthropic’s guide to building effective agents [9], apply across all patterns:

1. Keep it simple. Use the simplest architecture that works. Add complexity only when demonstrated necessary. A prompt chain that solves the problem is always preferable to a multi-agent system that might.
2. Transparency over cleverness. Every step should be inspectable. Avoid hidden state or implicit reasoning. When an agent fails, you need to understand why—opaque architectures make debugging impossible.
3. Provide good tools. Well-documented, well-typed tools with clear error messages are force multipliers. A tool with a vague description will be misused; a tool with a precise schema and usage guidance will be selected correctly.
4. Plan for failure. Every tool call can fail. Build retry logic, fallbacks, and graceful degradation at the harness level so the model does not need to reason about infrastructure failures.
5. Use structured outputs. Constrained generation (JSON schema, function calling) prevents parse failures. An agent that produces free-form text requiring regex parsing is fragile; one that produces validated JSON is robust.
6. Test with diverse inputs. Agent behaviour is more variable than single-turn chat. The same prompt can produce different tool-call sequences on different runs. Test adversarially, with edge cases, ambiguous requests, and malformed inputs.

Pattern Selection Guide

Choosing the right pattern depends on three factors: (1) how predictable the task structure is, (2) how many LLM calls you can afford in latency and cost, and (3) whether quality requires iteration. Use the table below as a decision matrix—start from the top (simplest) and move down only when the simpler pattern demonstrably fails.

Patterns are composable: a planning agent may use prompt chaining for individual steps, an evaluator-optimizer within its review phase, and routing to dispatch subtasks to specialists. The art is knowing when to stop adding layers.