Learning from Agent Feedback and Outcomes
Why orchestrators need to learn
Here's the thing, most production multi-agent systems start with static routing rules. Your orchestrator has a fixed decision tree: billing questions go to Agent A, technical issues to Agent B. That works until it doesn't. From what I've seen, the breakdown happens when edge cases pile up, when one agent becomes a bottleneck, or when context changes and your hardcoded rules drift out of sync with reality.
Learning from feedback means your orchestrator adapts its decisions based on what actually happened, which agents succeeded, which failed, which subtask decompositions led to hallucinations, which routing choices wasted time. You're closing the loop. The system observes outcomes, updates its strategy, and gets better over time without you rewriting rules every week.
This isn't about running full reinforcement learning models (though you can). In practice, most production systems use simpler adaptive mechanisms: tracking success rates per agent, adjusting routing weights with multi-armed bandit algorithms, or flagging error patterns that trigger human review. The goal is the same, turn execution history into smarter coordination.
Check your understanding
Your orchestrator routes customer support tickets to three specialized agents. You notice that Agent B (technical issues) has a 40% escalation rate to humans, while Agents A and C average 10%. What's the first feedback-driven action you should take?
What feedback looks like in practice
Feedback isn't abstract. It's observable signals from execution, task completion status, agent response times, error codes, human intervention flags, downstream validation failures. In a financial workflow, it might be "transaction agent returned success but verification agent caught a missing field." In a research pipeline, it's "reviewer agent rejected draft three times before approving." These are datapoints you can log, aggregate, and act on.
The trick is to instrument your orchestrator to capture these signals at handoff boundaries. When Agent 1 passes context to Agent 2, log the handoff timestamp, the state snapshot, and Agent 2's outcome. When a subtask fails, record which agent was responsible, what the error was, and how the orchestrator recovered (retry? escalate? reroute?). Chi Wang's work on conversational agents emphasized that dialogue turns are natural checkpoints for this, every agent response is a chance to evaluate quality.
From what I've observed, the most useful feedback combines immediate outcomes (did the agent return a valid response?) with delayed outcomes (did the customer mark the ticket resolved? did the transaction clear compliance?). Immediate feedback lets you catch errors fast. Delayed feedback tells you if your agents are actually solving the business problem.
Start with explicit validation checkpoints
Don't wait for end-to-end metrics. Add lightweight validation at each agent handoff, schema checks, null guards, expected-value ranges. If Agent 1 should return a transaction ID and it doesn't, flag it immediately. Mikiko Bazeley's research showed that validation checkpoints prevent hallucination propagation before bad data infects downstream agents.Check your understanding
You're building an e-commerce order orchestrator that coordinates inventory check, fraud detection, payment, and fulfillment agents. What are three specific feedback signals you would log to help the orchestrator learn which routing decisions work well?
Adaptive routing strategies
Once you're collecting feedback, you can adjust routing decisions dynamically. The simplest approach is a success-rate tracker: if Agent A solves 90% of billing tickets and Agent B solves 70%, route more billing traffic to A. You're not doing anything fancy, just counting outcomes and shifting probability weights.
A step up is a multi-armed bandit algorithm, borrowed from recommendation systems. Each agent is an "arm" with an unknown success rate. The bandit balances exploration (try agents you haven't used much to learn their performance) with exploitation (route to the best-performing agent based on current data). Epsilon-greedy and Thompson sampling are common variants. This is what most production systems mean when they say "the orchestrator learns", not deep RL, just a lightweight probabilistic policy that updates with every outcome.
For more complex scenarios, you can learn routing rules based on task features. If your orchestrator sees that tickets mentioning "refund" succeed 95% of the time with Agent A but only 60% with Agent B, it can route refund tickets preferentially to A. You're building a lightweight classifier (rule-based or logistic regression) that maps task characteristics to agent performance. Harrison Chase's LangGraph supervisor pattern supports this with custom routing functions that take state as input.
Don't over-optimize on early data
If you've only seen ten outcomes, a 90% success rate might be luck. Use confidence intervals or set minimum sample thresholds before shifting routing weights. I've seen orchestrators lock onto suboptimal agents because they got lucky in the first batch of tasks. Give your bandit algorithm enough exploration budget to learn the true performance distribution.Check your understanding
Learning what went wrong, not just that it failed
Here's where things get practical. Knowing Agent B failed 40% of the time is useful. Knowing why, missing context from the previous agent, task type mismatch, API timeout, is what lets you fix it. Structured error logs and failure taxonomies turn raw feedback into actionable insight.
In a climate data workflow studied by researchers, agents would fail silently when upstream data retrieval returned empty results. The orchestrator logged "task failed," but not "failed because no data matched the query." Once they added error classification (data unavailable, API error, schema mismatch, agent refusal), they could route retries intelligently, reroute data errors to a fallback scraper, retry API errors with backoff, escalate schema mismatches to a validation agent.
From what I've seen, the best production systems also track hallucination propagation. If Agent 1 hallucinates an API response format in step 2, and by step 5 the entire pipeline is operating on a fictional premise, your orchestrator needs to catch that early. Governed shared memory architectures (like those Mikiko Bazeley studied at MongoDB) add provenance tracking, each fact in shared memory records which agent asserted it and when. If downstream agents detect inconsistency, the orchestrator can trace it back to the source agent and either retry or flag for review.
Error patterns reveal architectural problems
If one agent consistently fails on a specific subtask type, that's a signal your task decomposition is wrong or the agent lacks the right tools. Adaptive routing can paper over the issue short-term, but the real fix is often to redesign the agent's capabilities or split the subtask differently. Let feedback guide your architecture, not just your routing weights.Key takeaways
- Orchestrators learn by logging execution outcomes, success rates, latency, validation failures, and downstream business metrics, at every agent handoff.
- Adaptive routing starts simple: track per-agent success rates and shift traffic toward better performers using multi-armed bandit algorithms or weighted probability.
- Structured error logs and failure taxonomies let you understand *why* agents fail, not just *that* they failed, so you can reroute intelligently or fix root causes.
- Validation checkpoints at handoff boundaries catch hallucinations and bad data before they propagate downstream and corrupt the entire workflow.
- Learning mechanisms must be explicitly designed, agents don't improve automatically; you build the feedback loop into your orchestration logic.
Your product check-in
Apply “Learning from Agent Feedback and Outcomes” to a product or workflow you know. What would you try, what could go wrong, and what evidence would help you decide?