Adaptive Orchestration: Dynamic Routing and Replanning

When static routing isn't enough

Here's the thing, most orchestration tutorials show you a neat diagram where Agent A always hands off to Agent B, which always routes to Agent C. That works until reality shows up: a worker agent times out, a subtask turns out to be more complex than anticipated, or your fraud-detection specialist is suddenly handling three requests at once.

Adaptive orchestration means your master agent adjusts in flight. It picks which worker to call based on capability match, current load, and past performance. When something fails, it doesn't just log an error, it reroutes, decomposes the task differently, or escalates. From what I've seen in production systems, the difference between a prototype and a reliable workflow is how it handles the unexpected.

Think of it like a project manager who doesn't just assign tickets blindly. They notice when someone's stuck, reassign work when priorities shift, and remember which team member is best at certain problem types. Your orchestrator can do the same, if you design it to learn and adapt.

Check your understanding

Your e-commerce orchestrator routes incoming orders to three worker agents: InventoryAgent (checks stock), FraudAgent (validates payment), and FulfillmentAgent (ships orders). During a sale event, InventoryAgent starts timing out due to high load. What adaptive routing strategy best maintains system reliability?

Dynamic routing: rules, LLMs, or both

You've got two main paths for deciding which agent handles which task. Rule-based routing uses explicit logic, if intent equals "billing", route to BillingAgent; if sentiment is negative and value over $10k, escalate to SeniorSupportAgent. It's fast, deterministic, and easy to debug. Harrison Chase's LangGraph supervisor pattern implements this with typed state channels that enforce routing rules at the graph level.

LLM-based routing is different. Your orchestrator looks at the task description and generates a routing decision on the fly, reasoning about which combination of agents makes sense. AG2's GroupChat pattern (from Chi Wang's original AutoGen work) uses a selector agent that decides who speaks next based on conversation context. It's flexible, handles edge cases you didn't anticipate, but costs more tokens and introduces latency.

From what I've observed in production, hybrid approaches win: use rules for the 80% of cases you can predict, and fall back to LLM reasoning for ambiguous requests. João Moura's CrewAI framework lets you define role hierarchies with fallback logic, so your manager agent uses heuristics first and deeper reasoning only when needed.

Key ideas: deciding which agent handles which task. generates a routing decision on the fly. hybrid approaches win — Key ideas from this lesson, grouped for review.

The coordination overhead trap

More agents don't automatically mean better performance. Every additional agent adds latency, token cost, and failure points. I've seen teams spin up seven specialized agents when two well-prompted ones would have finished faster. Use the simplest pattern that meets requirements, orchestration is infrastructure overhead, not free performance.

Check your understanding

Replanning when things go wrong

Let's get real: worker agents fail. APIs time out. External services return unexpected formats. An agent hallucinates a step in your twelve-part pipeline. The question isn't if you need replanning, it's how sophisticated your recovery needs to be.

Basic replanning is retry-with-backoff: the orchestrator catches the error, waits, and tries again (maybe with a different worker). Smarter replanning decomposes the task differently. If your ResearchAgent can't find a full climate dataset, maybe the orchestrator breaks it into two subtasks, one for temperature data, one for precipitation, and routes them separately. Mikiko Bazeley's MongoDB research on multi-agent memory showed that context checkpoints at each handoff let you roll back cleanly without restarting the entire workflow.

The most adaptive systems track why failures happen. If FraudAgent times out during high load, log that pattern. Next time load spikes, route fraud checks to a backup instance preemptively. You're not doing full reinforcement learning here (most production systems don't), just maintaining a simple ledger of what works and adjusting routing weights accordingly.

State management is your safety net

The ClimateAgent workflow shows this well: each agent appends its output to a shared, persistent context rather than passing ephemeral messages. When an agent fails mid-pipeline, you can replay from the last checkpoint instead of restarting from scratch. Think of it like Git commits for your workflow state, each handoff is a checkpoint you can roll back to.

Check your understanding

You're building a financial loan processing orchestrator. When the TransactionAgent fails to process a loan payoff in step 8 of a 12-step workflow, what architectural features would let you replan effectively without restarting the entire flow? Describe at least two mechanisms.

Learning from outcomes: closing the loop

Adaptive orchestration isn't just about recovering from failures, it's about getting smarter over time. Every workflow execution generates signal: which agent completed fastest, where handoffs introduced drift, which routing decisions led to escalations. If you capture that and feed it back, your orchestrator improves.

Most production systems don't use full reinforcement learning (too complex, too slow to converge). Instead, they use simple feedback mechanisms like multi-armed bandit algorithms. Track success rate and latency for each worker agent. When routing a new task, pick the agent with the best recent track record for that task type, with occasional exploration to discover if a previously slow agent has improved.

The customer support orchestration example from the research bundle shows this in action: a routing agent classifies tickets and dispatches to specialists, but it also logs resolution time and customer satisfaction. Over weeks, it learns that certain billing issues resolve faster when routed to the payments team rather than general support. You're building institutional memory into the system, not just executing fixed workflows.

Start with structured logging

Before you build adaptive routing, instrument your orchestrator to log every routing decision, worker selection, execution time, and outcome. You'll spot patterns manually first, like "FraudAgent always times out between 2-4pm" or "ComplexQueryAgent fails 40% of the time on requests over 500 tokens." Those patterns become your first routing rules.

Key takeaways

Adaptive orchestration adjusts worker selection, task decomposition, and recovery strategies based on runtime conditions and historical performance.
Hybrid routing, rule-based for common cases, LLM-based for ambiguous ones, balances speed, cost, and flexibility in production systems.
Replanning requires checkpointing and shared memory so you can roll back to validated states rather than restarting entire workflows when agents fail.
Learning from outcomes (success rates, latency, error patterns) lets your orchestrator improve routing decisions over time without complex reinforcement learning.
Coordination overhead is real, simpler patterns with fewer agents often outperform elaborate multi-agent systems that lack solid state management.

Your product check-in

Apply “Adaptive Orchestration: Dynamic Routing and Replanning” to a product or workflow you know. What would you try, what could go wrong, and what evidence would help you decide?