AI Agent Planning in Practice: Task Decomposition, Plan Validation, Replanning, and Failure Recovery

Who This Is For

• System architects moving from single-agent workflows to complex multi-agent collaboration.
• Frontline developers focused on communication latency, network overhead, and debugging logs for multi-agent systems in distributed environments.
• Technical decision-makers who need to evaluate the implementation value and cost of multi-agent applications in specific vertical business scenarios.

I. Deep-Water Reasoning: Why Your Agent Keeps Going Off Course

The hard part of LLM planning is not generating a plausible-looking to-do list. It is controlling an agent’s execution state and self-healing path when the agent encounters an unknown external environment.

In agent development, we often use this formula: Agent = Large Language Model + Planning + Memory + Tool Use. Planning is the brain’s navigation system. Without a reliable planning mechanism, an agent is merely an obedient but blind instruction executor. Once the task chain grows longer or a tool encounters an unexpected network disruption, the agent drifts logically or spins on the same error.

I’m Xiaobai. Late one night in my local development environment in Guiyang, I built an agent to audit financial research reports from multiple sources. At first, I thought I could simply tell the model, “First retrieve the financial reports of 5 competing companies, then compare them, write a 5,000-word research report, and email it to the manager,” and it would handle everything autonomously.

Reality taught me otherwise. In practice, the first company’s financial report was a corrupted PDF. After the LLM failed to parse it, the absence of plan-validation and replanning logic caused it to keep trying the same broken file until it exhausted my API token quota. At other times, it forgot all the data it had retrieved earlier and produced an incoherent summary full of nonsense. I learned that, in an industrial-grade deployment, planning must be a controllable, verifiable, and self-healing white-box reasoning loop.

II. Recommended Planning Topology: Production-Grade Agent Planning Architecture

A production-grade planning architecture must isolate goal parsing, step validation, task dispatch, and dynamic replanning into separate modules.

To keep the agent on course while it executes long task chains, I designed the planning engine around the following core topology:

用户总目标 (User Goal)
  │
  ▼
目标解析器 (Goal Parser)
  │
  ▼
任务拆解器 (Task Decomposer)
  │
  ▼
计划校验器 (Plan Validator - 防御性拦截)
  │
  ▼
状态管理器 (State Store) ◄─────────┐ (状态更新/回滚)
  │                                 │
  ▼                                 │
工具/动作选择器 (Tool Selector)      │
  │                                 │
  ▼                                 │
步骤执行器 (Step Executor) ─────────┼─► 重规划引擎 (Replanner - 异常时触发)
  │                                 │
  ▼                                 │
停止控制器 (Stop Controller) ───────┘ (熔断判断)
  │
  ▼
最终输出 (Final Answer)

In this topology, the user’s raw input is first parsed and decomposed into strongly typed substeps, or Task Steps. Before execution, the Plan Validator filters out invalid or high-risk actions. After each step runs, its result, or Observation, is written to the State Store. If the execution engine detects a serious tool error, it does not stop immediately. Instead, it passes the error context to the Replanner, which dynamically revises the remaining plan while preserving completed steps as Checkpoints.

III. Task Decomposition: Turning Unstructured Goals into Strongly Typed Steps

Task decomposition must produce structured JSON with dependencies, tool mappings, and risk controls, not a loose set of natural-language notes.

Many developers take a shortcut and ask the model to produce a bullet-point to-do list in its chain of thought (CoT). That becomes painful when the plan must be passed to another system for execution or checked for valid dependencies.

The correct approach is to use the LLM’s structured-output mode, such as JSON Mode or Structured Outputs, during task decomposition and represent the task as strongly typed substeps.

The following is an example of the structured step model I use in a Python planning engine:

from typing import List, Dict, Any, Optional
from pydantic import BaseModel, Field

class TaskStep(BaseModel):
    step_id: str = Field(..., description="子步骤唯一标识，如 step_1")
    step_goal: str = Field(..., description="本步骤的明确动作目标")
    required_tool: str = Field(..., description="执行本步骤需要调用的物理工具名称")
    input_arguments: Dict[str, Any] = Field(default_factory=dict, description="工具调用入参字典")
    expected_output: str = Field(..., description="本步骤预期回传的数据结构说明")
    dependency_id: Optional[str] = Field(None, description="本步骤所依赖的前置步骤 ID")
    risk_level: str = Field("low", description="本步骤的操作风险评级 (low, medium, high)")
    status: str = Field("pending", description="步骤状态 (pending, running, completed, failed)")

With a strongly typed step dictionary like this, the execution engine can perform extensive validation in advance. It can check whether a dependency_id refers to a step that has already completed successfully, verify that required_tool is registered in the current Tool Registry, and confirm that the argument types in input_arguments conform to the tool’s JSON Schema. These checks make the plan reliable at the code level.

IV. Plan Validation: Run Defensive Checks Before Tool Execution

Plan validation is the first line of defense against agents abusing high-risk privileges or executing invalid actions, and it must happen before tasks are dispatched.

After decomposing a long task chain, we must never send the plan directly to the Executor. During planning, an LLM can generate incorrect parameters because it does not fully understand a local tool’s description. A malicious prompt injection can also lead it to create high-risk steps, such as deleting the root directory or downloading sensitive files without authorization.

An independent Plan Validator must therefore apply the following mandatory checks to every TaskStep after plan generation and before execution:

1. Parameter Completeness and Format Review

The validator parses each step’s input_arguments and checks for missing required parameters, such as whether an order-details step actually extracted an order_id. If a parameter is missing, the validator rejects the plan locally and asks the LLM to extract it again. It does not call the database API, thereby avoiding pointless network latency.

2. Permission and High-Risk Action Blocking

If an LLM-generated step contains a high-risk operation, such as a risk_level of high or an attempt to write to a database, issue a refund, or send a group notification, the validator forcibly suspends execution and asks the control layer for human-in-the-loop (HITL) confirmation. This prevents the agent from completing dangerous actions autonomously.

3. Tool Availability Check

The validator checks whether the required_tool supplied by the LLM exists in the system’s allowlist. If the model hallucinates a nonexistent tool such as auto_reboot_server, the validator catches the error locally, returns an error message, and forces the LLM to self-correct and replan.

V. Execution and Reasoning Loop: Take a Step, Observe, and Update

Planning is not a one-time static workflow. It is a reasoning loop that uses real-time Observations to make dynamic corrections.

As LLM planning algorithms have evolved, static One-shot Planning has often performed poorly on long task chains because the physical environment is highly dynamic and uncertain. For example, the first step may be to retrieve a web page, but the page returns a 403 access error. A static plan will press ahead with data analysis in step two and ultimately produce a meaningless empty report.

Production systems therefore need a dynamic ReAct reasoning loop: Thought -> Action -> Observation -> State Update.

1. Thought: The model considers what to do next based on the global plan and current session state.
2. Action: It calls a local tool or executes a specific subtask.
3. Observation: The system captures the actual result returned by the tool, whether successful data, error text, or a timeout state.
4. State Update: The system appends the Observation to the State Store and updates the cache of completed steps.
5. Decision: The controller determines whether the task is complete because all steps finished, needs replanning because the current step failed with low confidence, or must trip a circuit breaker because it reached the maximum step count.

By introducing dynamic Observation feedback at every step, we place the LLM’s black-box reasoning on a track that code can intercept and observe in real time, greatly improving execution safety.

VI. Architecture Selection: ReAct, Plan-and-Execute, or Hybrid Orchestration

Depending on latency constraints and branching complexity, you must weigh ReAct, static planning, adaptive replanning, and Workflow approaches.

In real projects, different planning algorithms have dramatically different communication overhead and use cases. I have organized the common approaches into the following decision table for use during architecture design:

Planning Mode	Core Logic	Latency	Token Cost	Best Use Cases	Limitations
ReAct	Enforces a Thought-Action-Observation loop	High (calls the LLM at every step)	Medium to high	Exploratory tasks with few tool calls that require iterative diagnosis and correction, such as debugging	Prone to semantic loops; historical context can expand rapidly over long chains
Plan-and-Execute	Generates a structured plan first, then has the executor call each tool in sequence	Low (the execution phase requires little repeated reasoning)	Low	Tasks with clear goals and predictable processes that require fast responses, such as automatically generating research reports	Poor fault tolerance; cannot self-heal dynamically when a step fails during execution
Adaptive Replanning	An advanced form of Plan-and-Execute that invokes the LLM to revise the remaining plan after an execution failure	Medium	Medium	Medium- to large-scale complex task chains with unstable external APIs, such as automated reconciliation	Complex state management and demanding prompt design during replanning
Workflow + Agent	Humans define the high-level SOP in code, while agents plan dynamically within individual nodes	Very low (the process is fully controlled)	Very low	High-assurance scenarios such as enterprise approval flows and expense-report audits	Lacks fully autonomous flexibility and requires substantial routing code up front
Multi-Agent Planning (MAS)	Multiple specialized agents handle planning and cross-auditing at different levels	Very high	Very high	High-risk, highly complex scenarios requiring coordination across physical systems and toolchains, such as automated software engineering	Extremely difficult to debug; communication costs and latency grow exponentially along long chains

VII. Dynamic Replanning: Reconstructing the Path After Failure

When a tool times out or returns unexpected data, the agent must be able to revise its original plan from the latest context and recover on its own.

Tool-call failures are among the most common production problems. For example, an agent may plan to use search_google for current data, only for the search API to return a Rate Limit Exceeded error. In a basic agent demo, it may simply retry until it consumes every token. A production system with replanning should instead do the following:

1. Capture the actual error state returned by the tool.
2. Preserve the status of completed steps, such as step_1 and step_2.
3. Invoke the Replanner and send the LLM the failed step ID, the specific failure reason, such as API rate limiting, and the remaining task objective as context.
4. Have the LLM generate a revised plan under the new constraint—for example, abandoning search_google in favor of a local read-only database cache or calling search_bing as a fallback.
5. Load the new plan into the execution engine, move the execution pointer to the new step, and continue.

This ability to reconstruct the path after a failure is a genuine expression of agent autonomy. It is also key to raising the system’s success rate from 70% to more than 95%.

VIII. Stop Conditions and Hard Circuit Breakers: Prevent Infinite Loops

To prevent an agent from entering an infinite planning loop after encountering a logic flaw, the system needs strong circuit breakers for global step counts and resource budgets.

If an LLM receives unclear guidance during replanning, or if the new plan still depends on a broken tool, it can enter a Planner Loop: Generate plan -> Execution error -> Trigger replanning -> Generate the same plan -> Repeat the same error

To stop this uncontrolled token consumption decisively, the orchestration layer must enforce the following circuit-breaker metrics as Stop Controller conditions:

• Maximum execution-step limit (max_steps): No matter what the LLM tries, the system must trip the circuit breaker unconditionally once a task reaches 15 executed substeps.
• Consecutive same-tool failure limit (max_tool_failures): If the same tool fails more than 3 consecutive times, immediately mark it unavailable for the task and force a fallback path or human handoff.
• Maximum replanning count (max_replanning_count): Limit Replanning to 3 attempts per task.
• Maximum token budget (max_token_budget): Set the task’s maximum spend in the Billing Guard, such as a 100,000-token inference limit per task, and cut off network access immediately if the task exceeds it.

After stopping, the system should not merely throw a generic 500 error. It should return a structured diagnosis that identifies which steps completed, where and why execution became stuck, and what information or parameters the user must provide manually, giving a human operator clear audit material for taking over.

IX. Failure Recovery and State Rollback: Build a Self-Healing System with Checkpoints

A high-confidence planning system needs state rollback so an agent can return to the previous checkpoint after a temporary network disruption.

Data consistency is one of the hardest problems in executing long, complex plans. Consider this agent plan: Step 1: Create a new user in the local database (SQL Write) -> Step 2: Register an API Key with a third-party provider (Network Call) -> Step 3: Send the customer an activation email.

If the network call in step two times out, starting from the beginning during replanning or retrying will repeat the SQL Write in step one and create dirty duplicate data in the database.

To solve this problem, the planning engine must support state Checkpoints and rollback:

1. Checkpoint mechanism: Each time a substep executes successfully and its state is updated, the system saves a state snapshot for the current tenant session in the database, including the current variable dictionary and hashes of tool return values.
2. Local idempotency check: Every tool that performs a write, such as create_user, must implement idempotency in code. It checks a unique external business identifier and returns success without writing again if the record already exists.
3. State rollback: When a step fails and cannot be retried in place, the engine forces the execution pointer back to the last successful Checkpoint, clears any temporary dirty state created along the way, and starts Replanning from that healthy point. This protects data and prevents repeated failed retries from polluting the context.

X. Planning Evaluation Metrics

Planning evaluation must break performance down into multiple technical and business dimensions, including plan validity, tool selection accuracy, and replanning success rate.

To evaluate the planning engine after strengthening it with dynamic replanning and validation, we introduced the following metrics into the development process:

1. Technical Planning Metrics

• Plan validity rate (plan_validity_rate): The percentage of generated plans that pass the Plan Validator without formatting errors, missing tools, or missing parameters. Our production target is above 97%.
• Logic drift rate (loop_rate): The percentage of tasks in which the agent enters a planning loop or repeats the same invalid step, used to evaluate prompt alignment.
• Replanning success rate (replan_success_rate): The percentage of Replanning attempts that ultimately reconstruct a valid plan and achieve the user’s original goal. We currently reach 88% with Sonnet 3.5.
• Average steps per task (avg_steps_per_task): A measure of planning efficiency. Fewer steps indicate more focused planning and lower token costs.

2. Business Planning Metrics

• Final task completion rate (task_completion_rate): The percentage of tasks in which autonomous planning and execution ultimately achieve the user’s expected goal.
• Partial success rate (partial_success_rate): The percentage of tasks that successfully return completed intermediate results when a physical constraint prevents further execution. We favor partial success over a complete error.
• Average cost per task (cost_per_completed_task): The average amount in US dollars consumed to complete one complex planning task successfully.

XI. Common Design Mistakes and Error-Log Troubleshooting

A poorly designed planning engine is highly susceptible to logical drift, recursive explosion, and expensive infinite loops, so it needs precise safeguards.

While moving a planning engine into a high-concurrency production environment, I encountered countless painful implementation traps. Here are three representative error logs and the corresponding troubleshooting measures:

1. Logic Drift

• Symptom: After executing more than 8 steps, the agent completely diverges from the user’s original objective and starts spinning through irrelevant side-path tools.
• Error message:

  Warning: [LOGIC_DRIFT_DETECTED] Task 'task-1012' current thought similarity to raw user_goal 'Generate Q2 Budget Report' has fallen below threshold 0.40. Current focus: 'Translating read_file debug comments to French'.
  

• Root cause: As execution steps and Observation logs accumulate, detailed tool results fill the model’s context window. The LLM’s native Attention mechanism decays, causing it to forget the original Goal.
• Troubleshooting: In the prompt template for each round of the Reasoning Loop, force the user’s raw_user_goal to appear at the very bottom of the Prompt, close to the model output area. Also add an assertion during the Thought stage that checks the relationship between the current action and the user’s ultimate goal. If the cosine similarity between the current action and the original goal falls below the threshold, forcibly truncate the context, keep only the Task Steps skeleton and current Observation, and inject the main goal again.

2. Vague Observation Loop

• Symptom: A tool fails with a vague error such as Error 500. Because the Agent cannot determine the cause during the Thought stage, it mechanically repeats the same tool call.
• Error message:

  Error: [PLANNER_RETRY_LOCK] Agent repeated step_3 'query_postgres' 3 times with identical arguments. Reason: Tool returned 'Internal Server Error (500)'.
  

• Root cause: The local tools passed raw HTTP status codes or null-pointer exceptions directly to the model instead of wrapping errors with semantic context. Without enough clues, the LLM cannot self-reflect effectively or adjust its parameters.
• Troubleshooting: When writing Tool functions, wrap underlying exceptions and return clear, meaningful error descriptions. For example, instead of returning 500, return: Tool execution failed: The Postgres SQL query timed out because missing indexes caused a full table scan. Try adding a LIMIT or optimizing the WHERE clause. Only an Observation with this kind of technical diagnosis allows the LLM to produce a correct self-healing plan during Replanning.

3. Recursive Decomposition Explosion

• Symptom: The LLM over-decomposes a simple task, such as “send the user an email containing an invoice,” into dozens of tiny atomic steps. The Context overflows within seconds and produces a large bill.
• Error message:

  Fatal: [CONTEXT_OVERFLOW] Planning step count reached 24. Task decomposition generated nested sub-plans. Context window exceeded 128,000 tokens at step 'step_18_verify_cc_email_format'.
  

• Root cause: The decomposition Prompt does not constrain decomposition depth or step granularity, so the LLM becomes excessively meticulous and isolates every supporting format check as a separate step.
• Troubleshooting: Add strict constraints to the Task Decomposer prompt: each task must be decomposed into 3 to 8 steps, and nested subplans are prohibited. Implement basic checks, such as email-format and filename-validity validation, directly in local tool code instead of asking the LLM to plan them as independent steps.

XII. Continue Reading

To deploy high-confidence agents in production, you need to learn how to introduce robust governance rules at every level of the process.

• 👉 AI Agent Architecture in Practice: From Prompts to Production-Grade Agent Systems
• 👉 AI Agent Tool Use: A Practical Guide to Tool Calls and API Integration
• 👉 AI Agent Memory System: A Deep Dive into Long-Term and Short-Term Memory Design
• 👉 AI Agent Observability: A Guide to Opening the Agent Execution Black Box
• 👉 AI Agent Evaluation: Four Core Dimensions and Practical Metrics for Quantifying Agent Performance
• 👉 AI Agent Deployment: Production-Grade Architecture and High-Concurrency Scaling
• 👉 Multi-Agent Systems in Practice: Architecture Boundaries, State Handoffs, and Failure Control
• 👉 Multi-Agent Planning: An In-Depth Guide to Planning Strategies for Multi-Agent Collaboration
• 👉 AI Agent Frameworks Compared: LangChain vs. AutoGen vs. CrewAI
• 👉 Microsoft AutoGen Agents in Practice: A Progressive Development Guide
• 👉 MCP vs. A2A vs. Function Calling: Protocol Selection and System Integration for AI Agents
• 👉 The Complete AI Agent Guide (2026): A 100,000-Word Practical Handbook for Industrial-Grade Agent Systems

External references:

• LangGraph Plan-and-Execute design patterns
• AutoGen Reasoning and Dynamic Planning configuration
• Microsoft Semantic Kernel Planner standard specification
• OpenAI JSON Schema mode for Structured Agent Planning

Topic path / AI Agents

Continue from one agent pattern to the complete production system

The AI Agent hub organizes architecture, memory, tool use, evaluation, security, deployment and multi-agent coordination into a single learning path.

Open AI Agent hub → Review the architecture guide Subscribe for updates